Linear Surface Reconstruction from Discrete Fundamental Forms on Triangle Meshes We present a linear algorithm to reconstruct the vertex coordinates for a surface mesh given its edge lengths and dihedral angles, unique up to rotation and translation. A local integrability condition for the existence of an immersion of the mesh in 3D Euclidean space is provided, mirroring the fundamental theorem of surfaces in the continuous setting (i.e., Gauss's equation and the Mainardi-Codazzi equations) if we regard edge lengths as the discrete first fundamental form and dihedral angles as the discrete second fundamental form. The resulting sparse linear system to solve for the immersion is derived from the convex optimization of a quadratic energy based on a lift from the immersion in the 3D Euclidean space to the 6D rigid motion space. This discrete representation and linear reconstruction can benefit a wide range of geometry processing tasks such as surface deformation and shape analysis. A rotation-invariant surface deformation through point and orientation constraints is demonstrated as well. Local rigid motion invariant representations often serve as a starting point for differential geometry of surfaces. In the discrete setting, such representations are also of great interest in various geometry processing contexts. For geometric modeling, local encoding of geometric details facilitates intuitive surface manipulation and deformation. For the simulation of elastic thin-shell objects, the strain energy must only depend on invariants of rigid motions to ensure preservation of momenta. Even parameterization, geometric texture synthesis, shape analysis, or 3D model registration can potentially benefit from an efficient rotation-invariant description of the shape.. An obvious choice for such a representation for surface meshes would be to use edge lengths and dihedral angles, which uses approximately twice as much storage space as the original vertex coordinates, but is indeed rotationinvariant and local. Moreover, given an arbitrary parameterization, they can be used to construct the first and second fundamental forms, which describe the metric of the surface and the bending of normals respectively. However, constructing a surface embedding (or immersion) in a linear fashion from an arbitrary set of edge lengths and dihedral angles is far from straightforward-except when the set of values are known to come from an existing embedding, in which case a sequential reconstruction starting from a seed vertex is trivial.. In this paper we provide a (necessary and sufficient) local integrability condition for a given triangle connectivity to admit an immersion in R 3 for a set of prescribed edge lengths and dihedral angles. We also present a linear approach to determining an embedding even when there are conflicting point and orientation constraints, or when the edge lengths and dihedral angles fail to satisfy our integrability conditions.. Related Work With the increase in computing power and user demands, mesh deformation and other related geometry processing methods have become popular tools. Their goal is often to produce reasonable, natural-looking surfaces that satisfy user-specified deformation constraints at near interactive rates. There are mainly two types of algorithms: surface-based, and ambient-space-based (also called free-form) deformation methods. As our contribution fits squarely in the first category, we mainly go over related surface-based methods. Some among these methods focus on point sampled geometry in a global coordinate system and employ a scalar (blending) field to get a smooth transition between constrained and unconstrained parts. For instance, [BK03] uses geodesic distances to define the blending function and adds an implicit occluder function to the mesh editing environment, while [PKKG03] uses Euclidean distances in a hybrid geometry model combining implicit and parametric surface representations. The latter produces more intuitive deformation at the cost of increased computational time. A number of authors, seeking physically plausible results, propose to minimize various physicallymotivated deformation energies. For example, [CPSS10] defines an elastic energy based on the distance between the deformation and the rotation group, including an additional thin-shell bending term if necessary; [ The deformation technique that we derive from our discrete surface representation can be seen as a linear method within differential-coordinates methods. Among those, [LSLCO05] is the only existing linear and rotationinvariant approach (not including iterative methods with linear steps, of course). It defines a local frame per vertex and encodes the local geometric information into the transition between adjacent local frames. The method produces deformations that preserve local details in the least-squares sense through a linear solve for the local frames and another for the coordinates. It handles rotational constraints efficiently, but is insensitive to translational constraints, since the solve for the frames is decoupled from the positional constraints [BS08] . [LCOGL07] developed a mesh editing method that aims at preserving shape and volume based on this representation. Our method strives to achieve the same goal, but in sharp contrast, we provide a natural and compact description of the fundamental forms that does not rely on the existence of an original embedding, the choice of a local vertex normal, or the choice of picking an edge as the first tangential direction, and produces mesh deformation in a single linear system. [KG08] also proposed to attach one local frame for each triangle face, but the frames are not orthogonal and the linear system cannot be derived from given edge lengths and dihedral angles without an existing mesh. Edge lengths and dihedral angles have been used as the local shape descriptor in the physics-based nonlinear energy in [FB11] to generate natural looking mesh deformation. They are also used for geometry interpolation in [WDAH10], where computation of their interpolation before the vertex location interpolation was proposed to better preserve local shape. We have introduced compatibility conditions for edge lengths and dihedral angles for meshes with fixed topology, akin to Gauss's equation and the Codazzi equations. We have proposed a sparse linear system based on these lengths and angles, the solution of which produces a mesh immersed in 3D with edge lengths and dihedral angles close to the specified ones. We have also demonstrated the benefits of such a rotation-invariant representation for mesh deformation applications.. Our discretization mixes the face-based frame representation and intrinsic connection (ω One drawback of our approach (common to other work) is the lack of guarantee of frame orthonormality-although for certain applications, it can provide additional flexibility necessary to satisfy the constraints. We wish to examine the possibility of using quaternions to represent rotations with a nonlinear but tractable optimization process as in, e.g., [CPS11] . Like the differential surface theory it is based on, our method cannot handle intersections without resorting to other methods. However, this is also common to other methods in the same category, and can actually be remedied by incorporating existing collision handling techniques.. We have proposed an ad-hoc method in Sec. 2.3 for conversions between matrix functions and discrete fundamental forms, and we wish to further investigate the continuous limits of the matrix functions when we refine the mesh. Another topic worth exploring is the design of a modified version of our energy functional to serve as a potential energy for thin shell simulations: we could associate translational velocities at vertices with a lumped mass matrix, and rotational velocities at faces with moments of inertia of each triangle. We also wish to study proper handling of potential energy with inhomogeneous and anisotropic material property using our basic representation. Based purely on geometric information, we can also try to weight different dihedral angles differently to enforce feature preservation, or weight regions differently to modify their rigidity according to user specifications. We can also manipulate the Frobenius norm, and put different relative weighting, e.g., for rotations and translations. The integrability conditions we proposed can potentially be combined in local neighborhoods to allow for local permissible editing operations on the rotation invariant representation that does not influence other areas-this seems plausible as a perturbation in a vertex location only changes the edge lengths and dihedral angles of its one-ring. We also wish to explore the application in deformation transfer using a patched-based extension to our representation similar to the extension to [LSLCO05] in [BVGP09] . Last, we may , angle distortion for results using our method, edge distortion for results using LRI, and edge distortion for results using our method. The pseudo color denotes the absolute value of changes in edge length and dihedral angles measured in radian. The sizes of the bounding boxes for the bar model (with 1666 vertices) and the armadillo model (with 16K vertices) are 1 × 1 × 6 and 1 × 1.2 × 0.9 , respectively. try to enhance the efficiency of the algorithm through a multiscale approach, although it is unclear whether significant gains will be incurred since the large linear system we currently use is very sparse (each face is only adjacent to three other faces). The basic local rigid-motion invariant formulation for surface equation can potentially benefit various applications involving geometry processing using triangle meshes. We developed a few proof-of-concept examples. We ran all the tests on an Intel Core2Dual@3.0GHz CPU with 4GB RAM.. Deformation Our first example is mesh deformation or editing using position and (optional) rotation constraints. Using Region of Interest and prefactorization of the SPD matrix, we can achieve interactive rates for moderate size regions, similar to e.g. [KCVS98, SCOL * 04, LSLCO05]. See Figure. 1. We show some extreme test cases as in the survey paper [BS08] ( Figure. 10) in Figure. 3.. Quasi-isometric Parameterization One interesting example when we manipulate the fundamental forms is to set II to 0, thus flattening the surface patch. In some sense, it is striving to preserve the first fundamental form, thus it can be called quasi-isometric. This method represents a new linear method to produce a parameterization with a "natural" boundary (see [DMA02] ). In this case, orthogonality can be weakly enforced by a quadratic penalty term. We sometimes perform a second linear solve as for the extreme constraints, since the abrupt change in II leads to large deviations from the compatibility conditions. However, as such, it cannot guarantee an embedding in 2D, but just an immersion. In such cases, the target function still forces the frames to produce a planar mesh, but the frames may be turned lefthanded, i.e., flipped from the original orientation for certain triangles. However, parameterization of typical meshes rarely gives rise to flips, as illustrated in Figure. 5.. Independence to Sampling Density In Figure. 2, a torus with varying vertex densities was deformed. As shown from the results, the overall shape remains the same despite the abrupt change in sampling density, and the symmetry of the shape is preserved. Comparison to Existing Work There are a number of differences between our method and the only existing linear and rotation-invariant approach [LSLCO05]. First, our discrete fundamental forms representation is (around 50%) more compact, independent of the choice of vertex normal, and independent of choice of the edges in the definition of frames (as we have proven mathematically, and observed numerically). Second, we do not require a prior embedding of the triangle mesh. Third, we couple the rotation and translation constraints in a natural fashion. As shown in Figure . 4, our method produces similar results measured by Figure 3 : Test cases used in [BS08] . From left to right: bumpy plane before and after frame orthonormalization, 120. • bent cylinder before (top) and after (bottom) orthonormalization, 135. • twisted bar before and after orthonormalization, 70. • bent cactus. Note that the results with orthonormalized frames are produced by a second linear solve using normalized columns of the frames in the first solve as soft constraints to handle these extreme cases. In these extreme deformations, the second solve produces visible differences except for the cactus case. In the bumpy plane test before normalization, the smooth transition between frames would force the bumps to point upward as the constraints force the frames at both handle regions to point upward.. edge length change, but produces less angle distortion, especially for the more complicated armadillo model. We added compatible rotation constraints for both methods, so that the linear rotation-invariant (LRI) coordinates [LSLCO05] can produce results, although our method can in fact work with only translational constraints. For these large deformations, we found orthonormalization of the frames improves the results for both methods. We used a weight of 100 for the afore-mentioned soft constraints approach for orthonormalization of the frames in our method. Other surface deformation methods with point constraints are either not rotationinvariant or non-linear. For a comparison to these methods, refer to [BS08] (Figure. 10) for models deformed under same type of constraints (bumpy plane, cylinder bending, bar twisting, and cactus bending) as in Figure. 3. We briefly go through the concepts involved in the continuous fundamental theorem of surfaces, roughly following the exposition of [Fra03] (Sec. 8.5). For a more detailed treatment on the subject see, e.g., [IL03] (Chapter 2).. For a surface M with a parameterization (u 1 , u 2 ), if there is a local embedding x in R 3 , a first fundamental form (metric tensor) I can be induced by I αβ = x,α, x ,β , and a second fundamental form II by II αβ = x ,αβ , N , where N denotes the normal, lowercase Greek letters are used for indices ∈ {1, 2}, and an index after a comma in the subscript denotes partial derivatives. Intuitively, I measures lengths on the surface induced by the 3D Euclidean metric, and II measures the rate of change of the normal field. We can use I or its inverse to raise or lower indices for tensors defined on the surface. We use Einstein's summation convention in the following discussion.. Gauss's surface equations express the derivatives of the tangent vectors x,α, in the local frame formed by {x ,1 , x ,2 , N} as. is the Christoffel symbol of I. Note that if we treat the vector x ,αβ as a linear combination of the basis vectors in the local frame, all the coefficients (i.e., Γ γ αβ and II αβ ) in the above equations can be derived from I and II. Roughly speaking, given a symmetric positive definite matrix function I and a symmetric matrix function II, if we start from a seed point and a chosen pair of tangential directions x ,1 and x ,2 , we can propagate these directions along any paths on the surface by integrating these equations. We can then construct an immersion by integrating x ,1 and x ,2 .. I and II provide 6 numbers per point while the 3D immersion requires only 3 numbers (coordinates) per point, so there must be 3 equations per point that I and II must satisfy. In fact, as the left-hand side of the above equation describes the gradient of x,α, it has to be curl-free, which means. where R denotes the Riemann curvature tensor determined by I, and ∇ denotes the covariant derivative defined by I. The vanishing of the coefficients in front of x,τ (resp., N) is called Gauss's equation (resp., the Mainardi-Codazzi equations). Note that on surfaces, R τ αγβ has only one independent component R 1 212 ; thus Gauss's equation simply states that the Gaussian curvature det(I −1 II) is the intrinsic curvature.. The fundamental theorem of surfaces states that Gauss's equation and the Codazzi equations are not only a necessary condition for I and II to be induced by a local embedding, but also a sufficient one, as proven using the Frobenius theorem; see e.g. [IL03] . Intuitively speaking, the Codazzi equations guarantee that the propagation of the normal direction (determined by the two tangential directions) from a seed point is independent of the choice of paths one takes, while Gauss's equation guarantees that the rotation in the tangent plane (defined by the two tangent directions) is consistent with the parallel transport prescribed by the metric I, and thus also path-independent. The equations are seen as integrability conditions, as we may regard the surface embedding as the integral of I and II, which are the rigid motion invariants of the first and second derivatives of the embedding. We will prove our discrete version of the fundamental theorem of surfaces based on these observations next. The above convex energy function, together with possible constraints, can be optimized through a linear system resembling a Poisson equation. If we arrange elements of f in row-major order, we can express the sparse symmetric linear system using a block matrix; for example, the diagonal entry for the n-th row for x would be wnnId, and the block on the i-th row and j-th column for f would be. We omit further discussions here, as all the coefficients can be deduced from the energy functional. Through its similarity to the Poisson equation, it is easy to see that this block matrix is positive-definite once we fix at least one position and one rotation (or other equivalent combinations of constraints) as mentioned above.. The main procedure for our algorithm is 1. Compute R i j and a from I and II or directly from immersion when it is available. 2. Include hard or soft constraints for positions and (optionally) for rotations through variable elimination or penalty terms. 3. Solve the sparse linear system to get the immersion x. We do not enforce (and in fact cannot enforce with a linear system) that the frames stay orthonormal. However, as noted in [LSLCO05], since the R i j 's are orthonormal, f remains close to being orthonormal, except for extreme constraints (in which case, one may (ortho)normalize the resulting f , and solve the system again with the normalized frames as soft constraints. In this optional step, the penalty weight on a frame for deviating from the orthonormal frame in the first solve is set to 1, 000 in our tests. A larger penalty can lead to better preservation of the first fundamental form, but may also produce small discontinuity near constrained vertices ). Due to the fact that the frames are stored on triangles, we can also easily derive constraints for f if needed, obtaining the face normal N by the normalized cross product of two edges for triangles with all three vertices constrained.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Continuous learning of compiler heuristics Optimizing programs to exploit the underlying hardware architecture is an important task. Much research has been done on enabling compilers to find the best set of code optimizations that can build the fastest and less resource-hungry executable for a given program. A common approach is iterative compilation, sometimes enriched by machine learning techniques. This provides good results, but requires extremely long compilation times and an initial training phase lasting even for days or weeks.We present long-term learning, a new algorithm that allows the compiler user to improve the performance of compiled programs with reduced compilation times with respect to iterative compilation, and without an initial training phase. Our algorithm does not just build good programs: it acquires knowledge every time a program is compiled and it uses such knowledge to learn compiler heuristics, without the need for an expert to manually define them. The heuristics are evolved during every compilation, by evaluating their effect on the generated programs. We present implementations of long-term learning on top of two different compilers, and experimental data gathered on multiple hardware configurations showing its effectiveness. Building an exact model of modern computer architectures that a compiler can use to decide what optimizations algorithms to apply and what values to assign to her numerical parameters is impractical. Therefore, all compilers use heuristics to make such decisions, based on program-dependent values, called features.. Usually, compiler experts write the heuristics. This is a time-consuming task, and it is likely to yield suboptimal results since it is entirely based on the skill of one or few experts. Furthermore, in order to obtain the best results, the process of defining the heuristics should be repeated for every new architecture the compiler has to target.. In order to face these issues, researchers proposed various techniques. Iterative compilation [Kisuki et al. 2000 ] is the simplest one. It is based on generating hundreds of versions of the program being compiled, then running them while gathering execution times. The fastest version is then kept as the final result of the compilation. The M. Tartara was partially supported by a Roberto Rocca Doctoral Fellowship, awarded by "Fondazione Fratelli Agostino ed Enrico Rocca" Lugano, CH. Authors' address: M. Tartara (corresponding author), and S. Crespi Reghizzi, Dipartimento di Elettronica e Infomazione, Politecnico di Milano, Italy; email: mtartara@elet.polimi.it. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested fromvarious versions are created by applying a different compiler configuration for each of them. Depending on the specific iterative compilation algorithm considered, each configuration can determine the set of optimizations to be used, the order in which the optimization algorithms are applied, or some compilation parameters (e.g., loop unrolling factor, size of the tiles for a matrix tiling algorithm, limits for function inlining, etc.). The choice of which configurations, to try is determined by random selection. The overhead introduced by iterative compilation is too high for most applications, but the technique is useful in some specific areas, such as embedded systems [Bodin et al. 1998 ], where the available resources are scarce and a program is likely to be deployed on many identical systems, making the extra compilation time worth spending.. Machine learning compilation techniques have been introduced to restrict the configurations' search space [Kulkarni et al. 2004; Park et al. 2011 ]: a training phase is performed when the compiler is deployed, gathering data that are used to train a model. Whenever a new program needs to be compiled, the model is queried to predict good configurations. Usually, this just aims at focusing the selection of the configurations to be tested through iterative compilation on more promising areas [Agakov et al. 2006 ], but a similar approach can be used to predict a single configuration to be used as the result of the compilation [Fursin and Temam 2010] . Unfortunately, the training phase for building a good model is really long, up to several weeks [Fursin et al. 2008] , and this limits the applicability of machine learning approaches.. The contributions of this article are as follows.. -We introduce long-term learning, a new learning algorithm for compilers able to build a model of the target system while compiling programs. The model is used to optimize the programs while avoiding both the long compilation times of iterative compilation and the initial training phase of machine-learning-based approaches. -We show that, starting without knowledge of the target system, long-term learning is able to create heuristics at least as good as those of existing compilers (-O3 for GCC and the default optimization level for PetaBricks). We also show that, starting from such optimization levels, our algorithm is able to improve them. -We show that long-term learning is able to continuously improve the performance of a compiler online 1 , whereas most compiler learning algorithms work offline and require a long initial training phase. -We show that different compilers can benefit from the use of long-term learning, by presenting two implementation and experimental results gathered on multiple hardware configurations.. The rest of this article is organized as follows: Section 2 describes the long-term learning compilation algorithm, Section 3 describes our implementations of the algorithm and the setup of the experimental campaign, Section 4 presents the results of the experiments. Related work is presented in Section 5 and possible future improvements are discussed in Section 6. Section 7 concludes. Various directions for improvements are possible, and we mention just a few.. Sometimes, different compiler configurations applied to the same source file might lead to identical binaries [Kulkarni et al. 2004] . Moreover, since long-term learning is a multi-generation learning algorithm and uses elitism, it is likely that some candidates will be present, identical, in multiple generations. To prevent testing the same binary candidate more than once, we might compute a hash of the binary file, reusing previously recorded execution times where available. The default settings of our long-term learning implementation test a total of 18 candidates (as explained in Section 4.1) during a compilation. The timing runs are the slowest step of our algorithm, so, avoiding even just one of them would sensibly speed up the compilation process.. Currently, the selection of what program features to use in heuristic formulas is done randomly. A statistical approach computing the correlation between features used and the obtained speedup is likely to perform better. Also, adding the ability to compute more static program features, and the ability to use dynamic features derived from performance counters, might improve the quality of the heuristics.. Currently, long-term learning always tests a fixed number of candidate configurations. Although low, this number still might cause quite long compilation times. Using program features, we might determine whether many similar programs have been compiled before. We could then trust the current heuristics and just generate the best candidate accordingly (as in Long [2011] ), or, if the program is completely different from anything known, the full set of 18 candidates should be tested. Intermediate configurations (for example with just 1 or 2 generations instead of 3) could also be used when only some data is available from previously compiled similar programs.. For this article, we decided to use GCC because, being at the heart of PetaBricks, we could share part of the implementation effort for the two compilers. LLVM [Lattner and Adve 2004] could be another interesting compiler on which to implement longterm learning, as it provides a wide variety of optimization passes and the possibility to write a pass manager to decide which ones to execute using our algorithm. This article presented long-term learning, a novel algorithm to determine the best set of heuristics that a compiler can use to make decisions about which optimizations to enable and what values to assign to parameters, without any human intervention.. Experimental results on multiple architectures confirm that, when implemented in the GCC and PetaBricks compilers, the algorithm is able to obtain good results starting from no knowledge at all about good optimizations, reaching and sometimes surpassing the performance of the respective maximum optimization levels.. We also verified that when some initial knowledge is provided by the compiler writer, the algorithm is able to improve the performance, sometimes marginally, sometimes in a significant way. By analyzing more static and dynamic features we could improve the ability of the system to characterize the program being compiled, thus further increasing the quality of the result and the obtainable speedups.. Long-term learning improves the compiler by using the information it gathers from the execution time of the compiled programs. In order to compute it, it needs input data for the programs to be provided. Therefore, the ideal use of a long-term learning compiler is probably inside a test-driven development workflow, where the test data are prepared before the program itself and are already available during the compilation.. Long-term learning is meant to learn compiler heuristics, therefore, it is not strictly required for it to always be active. It allows the user to obtain compiler heuristics targeted at producing good programs for the architecture they will actually run on, not just good programs obtained by the iterations performed while the algorithm is active. Therefore, once satisfying heuristics have been learned, the long-term learning algorithm can be completely disabled, leaving a fast compiler producing optimized programs. Also, given two identical systems, the heuristics learned on one system can be ported on the other immediately giving good results on new programs. Many researchers have worked towards building compilers without the need for compiler programmers to manually define whether to apply each optimization algorithm. Agakov et al. [2006] show that it is possible to learn a model that predicts good optimizations based on the analysis of static program features. Experimental results from Cavazos et al. [2007] suggest that using performance counters instead of static features allows to further reduce the number of configurations to test.. Another approach was proposed by Dubach et al. [2007] . Every time they need to compile a program, they build a model by compiling and testing a small number (as low as 8) of versions of the program. Then, they use this model to predict-without execution-the performance of 500 randomly generated points of the search space. Up to 100 of the best points from this set are executed to find the actual best version. The problem with this approach is the need to build a program-specific model, and the fact that it does not reuse knowledge acquired from previous compilation. Thomson et al. [2009] try to reduce the training time by focusing it on the programs that best characterize the search space of the optimization sets. They first gather the static features of all the programs in the training set, then apply unsupervised clustering in the program feature space. This way, a classification of the programs according to the similarity of their features is obtained. The most typical program of each cluster is chosen, obtaining a subset of programs that is representative of the whole search space. Each of the programs of the reduced training set is then compiled and tested with 4000 different optimizations sets, using a supervised learning technique to learn the model. The model is then used to immediately provide the supposedly best configuration, one-shot, without using iterative compilation.. The workflow proposed by Long [2011] uses a reverse k-nearest neighbors classifier to determine if the program being compiled is similar to a previously met one. If it is, the best configuration that was recorded for the similar program is used. If the program is an outlier, iterative compilation with random search is used to search for good configurations. Information about all the tested candidate configurations is stored. This approach is similar to long-term learning because it does not need a training phase and reduces the number of times iterative compilation has to be used. On the other hand, when it is used, hundreds of executions are needed.. The main differences between long-term learning and other approaches are as follows. Most works just aim at improving the performance of the compiled programs.. Long-term learning, on the other hand, also aims at finding the best compiler heuristics expressed through human-readable formulas, that are easier to understand for compiler writers. To the best of our knowledge, the only other works with the same aim are those by Stephenson et al. [2003] , using mathematical formulas, and Monsifrot et al. [2002] , using decision trees. Furthermore, most works learn the single heuristics independently, whereas we also score sets as a whole, capturing the interactions between heuristics. Finally, most works just deal with changing command line flags, whereas our approach is more integrated in the compiler and can change internal parameters of the algorithms as well, as shown with PetaBricks tiling algorithm. In order to show the viability of our approach and its applicability to multiple compilers, we implemented long-term learning into two different compilation toolchains (GCC and PetaBricks), and performed tests on multiple hardware configurations.. The long-term learning algorithm implementation is divided into two parts. The learning algorithm itself is implemented in Python as a framework, and is common to all the compilers. It chooses and learns the heuristics. The second part is implementation specific and it gives the compiler a way to provide the program features to the algorithm and to use the chosen heuristics to make decisions about the compilation process. The compiler-specific implementations will be described in Section 3.1 and 3.2. In these experiments, most of the times the heuristics are then used to decide the best command line parameters, but this is just one possible use: in fact, the heuristic for the Petabricks tiling algorithm is evaluated directly inside the compiler itself. s e e F i g u r e 4 mutation probability (m) 0 . 3. As it can be seen in the Configuration section of Algorithm 1, long-term learning needs a few parameters to be set in order to define its behavior. We determined the values to assign to such parameters (in Table I ) through a brief experimentation.. Our implementations were tested on different hardware configurations, to show that the algorithm is able to adapt compilers to multiple architectures.. Hardware Configuration 1. A machine with 4 Intel Xeon X7550 processors (8 physical cores each) running at 2.00 GHz with 128GB of RAM.. Hardware Configuration 2. One core of a machine part of the CILEA Lagrange cluster [Arlandini and Invernizzi 2008] . Each machine is equipped with two Intel Xeon X5460 processors (4 physical core each) running at 3.16 GHz and 16GB of RAM.. Hardware Configuration 3. One processor of a machine equipped with two Intel Xeon X5460 processors (4 physical cores each) running at 3.16 GHz and 8GB of RAM.. We want to show that the knowledge acquired by the compiler through long-term learning can be applied to improve the performance of new programs. Therefore, we need to simulate the life of a compiler that has compiled a certain number of programs previously, showing how this affects the performance of an unseen program.. We have a set of available programs, taken from a benchmark suite. For every simulation, we use one of the programs of the suite as the test program and the rest as the training set, in a Leave-One-Out Cross-Validation [Kohavi 1995] configuration.. Each simulation is performed as follows. First of all, the test program is compiled using the default compiler configuration to find out the reference speed. Then, a program from the training set is compiled. Then, the test program is compiled again, using the knowledge acquired by the compilation of the first training program, and is then tested recording its execution time. Later, we compile the second program from the training set and then again we compile and run the test program, using the knowledge acquired from the compilation of the first two training programs. This procedure is repeated for each program in the training set.. It should be noted that, in order for the experiment to be meaningful, it is mandatory that the test program has never been seen by the system before, to prevent the compiler from acquiring knowledge from its compilation. In the simulation setup that was just described, the test program is compiled multiple times. In order to allow this, a special option was implemented in the compilers, enabling a program to be compiled using the current knowledge, but disabling write access to the knowledge base. Such option is used every time the test program is compiled, therefore, every time, the system behaves as if it never saw such program before.. It is also worth noting that we use the term "training set", but this does not mean that a training phase is needed: it just denotes the set of previously compiled programs, that provide the data for the learning process. In a production system, such programs would be the programs compiled by the compiler during its lifetime, up to the current one. The compiler is immediately usable, without an explicit initial training phase. Since the long-term learning algorithm performs many operations randomly, it is expected that two different simulations will yield different results. Therefore, in order to provide more statistically relevant information, all the graphs shown in Section 4 are plotted using data computed as the average execution time computed over 5 runs.. The next two sections will describe compiler-specific details of our implementations.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Band Structure Optimization Of Two-Dimensional Photonic Crystals In H-Polarization Abstract. In the study of photonic crystals, the question arises naturally: which crystals produce the largest band gaps? This question is investigated by means of an optimization-based evolution algorithm which, given two dielectric materials, seeks to produce a material distribution within the fundamental cell which produces a maximal band gap at a given point in the spectrum. The case of H-polarization in two-dimensions is considered. Several numerical examples are presented.

O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning complex action models with quantifiers and logical implications Automated planning requires action models described using languages such as the Planning Domain Definition Language (PDDL) as input, but building action models from scratch is a very difficult and time-consuming task, even for experts. This is because it is difficult to formally describe all conditions and changes, reflected in the preconditions and effects of action models. In the past, there have been algorithms that can automatically learn simple action models from plan traces. However, there are many cases in the real world where we need more complicated expressions based on universal and existential quantifiers, as well as logical implications in action models to precisely describe the underlying mechanisms of the actions. Such complex action models cannot be learned using many previous algorithms. In this article, we present a novel algorithm called LAMP (Learning Action Models from Plan traces), to learn action models with quantifiers and logical implications from a set of observed plan traces with only partially observed intermediate state information. The LAMP algorithm generates candidate formulas that are passed to a Markov Logic Network (MLN) for selecting the most likely subsets of candidate formulas. The selected subset of formulas is then transformed into learned action models, which can then be tweaked by domain experts to arrive at the final models. We evaluate our approach in four planning domains to demonstrate that LAMP is effective in learning complex action models. We also analyze the human effort saved by using LAMP in helping to create action models through a user study. Finally, we apply LAMP to a real-world application domain for software requirement engineering to help the engineers acquire software requirements and show that LAMP can indeed help experts a great deal in real-world knowledge-engineering applications. Automated planning systems achieve goals by producing sequences of actions from the given action models that are provided as input [14] . A typical way to describe the action models is to use action languages such as the Planning Domain Definition Language (PDDL) [13, 11, 14] in which one can specify the precedence and consequence of actions. A traditional way of building action models is to ask domain experts to analyze a task domain and manually construct a domain description that includes a set of complete action models. Planning systems can then proceed to generate action sequences to achieve goals.. However, it is very difficult and time-consuming to manually build action models in a given task domain, even for experts. This is a typical problem of the knowledge-engineering bottleneck, where experts often find it difficult to articulate their experiences formally and completely. Because of this, researchers have started to explore ways to reduce the human effort of building action models by learning from observed examples or plan traces. Some researchers have developed methods to learn action models from complete state information before and after an action in some example plan traces [4, 15, 31, 50] . Others, such as Yang et al. [51, 39] have proposed to learn action models from plan examples with only incomplete state information. Yang et al. [51, 52] have developed an approach known as Action Relation Modeling System (ARMS) to learn action models in a STRIPS (STanford Research Institute Problem Solver) [10] representation using a weighted MAXSAT-based (Maximum Satisfiability) approach. Shahaf et al. [39] have proposed an algorithm called Simultaneous Learning and Filtering (SLAF) to learn more expressive action models using consistency-based algorithms.. Despite the success of these learning systems, in the real world, there are many applications where actions should be expressed using a more expressive representation, namely, quantifiers and logical implications. For instance, consider the case where there are different cases in a briefcase 1 planning domain, such that a briefcase should not be moved to a place where there is another briefcase with the same color. We can model the action move in PDDL as follows. 2 action: move(?c1 -case ?l1 ?l2 -location) pre:. (:and (at ?c1 ?l1) (forall ?c2 -case (imply (samecolor ?c2 ?c1)(not (at ?c2 ?l2))))) effect:. (:and (at ?c1 ?l2) (not (at ?c1 ?l1))). That is, if we want to move the case c1 from the location l1 to l2, c1 should be at l1 first, and every other case c2 whose color is the same with c1 should not be at l2. After the action move, c1 will be at l2 instead of at l1. Likewise, consider a pilot could not fly to a place where there are enemies. We can model the action model fly as follows.. action: fly(?p1 -pilot ?l1 ?l2 -location) pre: (:and (at ?p1 ?l1) (forall ?p2 -person (imply (enemy ?p2 ?p1)(not (at ?p2 ?l2))))) effect:. (:and (at ?p1 ?l2) (not (at ?p1 ?l1))). We can see that in these examples, we need universal quantifiers as well as logical implications in the precondition part of the action to precisely represent this action and compress the action model in a compact form.. As another example, consider a driver who intends to drive a train. Before he can start, he should make sure all the passengers have gotten on the train. After that, if there is a seat vacant, then he can start to drive the train. We represent this drive-train action model in PDDL as follows. We use Markov Logic Networks (MLNs) [35, 9, 23 ] to help learn the action models in our work. MLN is a powerful framework that combines probability and first-order logic. An MLN consists of a set of weighted formulas and provide a way to soften hard constraints in first-order logic. The main motivation behind MLNs to "soften" these hard constraints is that when a world violates a formula in a knowledge base, it is less probable, but not impossible. Thus, each formula should have an associated weight that reflects how strong a constraint is. Each weight can be learned from data using a variety of methods, including convex optimization of the likelihood or a related function, iterative scaling and margin maximization [27] . The network structure can also be learned from the input data, typically by performing a greedy search over conjunctions of variables [21, 28, 20] . Furthermore, different versions of MLNs have been proposed. For instance, Wang and Domingos [43] introduce hybrid MLNs, in which continuous properties and functions over them can appear as features; Singla and Domingos [41] extend Markov logic to infinite domains by casting it in the framework of Gibbs measures. In this paper, we have presented a novel approach to learn action models with quantifiers as well as logical implications, from a set of observed plan traces where we can support partially observable intermediate states. Our LAMP learning algorithm makes use of Markov Logic Networks to learn action models automatically. Our empirical tests in four planning domains show that our LAMP algorithm is rather effective. We list several possible directions which we could follow for our future work. Our current LAMP algorithm will enumerate all the possible preconditions and effects (or candidate formulas) according to our specific correctness constraints. When there are many actions and predicates in a planning domain, the candidate formulas will be very many, which can decrease the efficiency of LAMP. In the future, we will consider how to incorporate more domain knowledge to filter out some "impossible" candidate formulas beforehand to make the algorithm much more efficient. Domain analysis can also help reduce the errors by providing a priori what conditions are important from a domain expert's point of view. Another direction is. to improve the quality of weight learning in a MLN. Currently we adopt a generative learning approach in LAMP, where we maximize the weighted pseudo log-likelihood. Other weight learning approaches such as discriminative learning might have some additional advantages. A third direction is to extend action model learning to learn elaborate action representations, including resources and functions. Finally, we will consider plan traces that contain false observations on actions and states and ways to filter out noise from the training data. In this subsection, we will summarize the major findings from our experimental results.. I In our experiments, we first validated the performance of our LAMP algorithm by varying the proportion of observed intermediate states. From Fig. 1 , we can see that our algorithm performance does not vary significantly when we change our observed state numbers from 100% to only 20%. In general, the more intermediate states we observe, the better performance we have. When we validate this experiment from a state perspective, i.e., we only observe a proportion of propositions, we get similar experiment results. We also vary the percent of propositions true in each state, and obtained similar results.. III From Figs. 1 and 2 , we notice that some experimental domains, such as elevator, are more difficult to learn than other domains. This may be because domains such as elevator have actions that contain more conditional effects than others, and these effects are more difficult to learn.. IV In our third experiment, we validate that the efficiency of LAMP decreases when we have more plan traces (refer to Fig. 3 ). We also record the running time we use when different numbers of plan traces are given as input. This running time information is used to fit a curve which empirically shows that LAMP has a polynomial time complexity (refer to Fig. 5 ) with respect to the number of plan traces.. V We also show our LAMP algorithm can help reduce the human efforts on creating action models, which can be seen from Fig. 6 . Furthermore, we give an example application of our LAMP algorithm (refer to Example 8) to show that LAMP can be applied to real-world domains, such as software requirement engineering.

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Top-K Structural Diversity Search in Large Networks Social contagion depicts a process of information (e.g., fads, opinions, news) diffusion in the online social networks. A recent study reports that in a social contagion process the probability of contagion is tightly controlled by the number of connected components in an individual's neighborhood. Such a number is termed structural diversity of an individual and it is shown to be a key predictor in the social contagion process. Based on this, a fundamental issue in a social network is to find top-k users with the highest structural diversities. In this paper, we, for the first time, study the top-k structural diversity search problem in a large network. Specifically, we develop an effective upper bound of structural diversity for pruning the search space. The upper bound can be incrementally refined in the search process. Based on such upper bound, we propose an efficient framework for top-k structural diversity search. To further speed up the structural diversity evaluation in the search process, several carefully devised heuristic search strategies are proposed. Extensive experimental studies are conducted in 13 real-world large networks, and the results demonstrate the efficiency and effectiveness of the proposed methods. Recently, online social networks such as Facebook, Twitter and LinkedIn have attracted growing attention in both industry and research communities. Online social networks are becoming more and more important medias for users to communicate with each other and to spread information in the real world [14] . In an online social network, the phenomenon of information diffusion, such as diffusion of fads, political opinions, and the adoption of new techniques, has been termed social contagion [22] , which is a similar process as epidemic diseases.. Traditionally, the models of social contagion are based on analogies with biological contagion, where the probability that a user is influenced by the contagion grows monotonically with the number of his or her friends who have been affected already [9, 3, 24] . However, such models have recently been challenged [19, 22] , as the social contagion process is typically more complex and the social decision can depend more subtly on the network structure. Ugander et al. [22] study two social contagion processes in Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Articles from this volume were invited to present their results at The 39th International Conference on Very Large Data Bases, August 26th -30th 2013, Riva del Garda, Trento, Italy. Proceedings of the VLDB Endowment, Vol. 6, No. 13 Copyright 2013 VLDB Endowment 2150-8097/13/13... $ 10.00. Facebook: the process that a user joins Facebook in response to an invitation email from an existing Facebook user, and the process that a user becomes an engaged user after joining. They find that the probability of contagion is tightly controlled by the number of connected components in a user's neighborhood, rather than by the number of friends in the neighborhood. A connected component represents a distinct social context of a user, and the multiplicity of social contexts is termed structural diversity. A user is much more likely to join Facebook if he or she has a larger structural diversity, i.e., a larger number of distinct social contexts. This finding reveals that the structural diversity of a user is an important factor in the social contagion process. As suggested in [22] , the analysis of structural diversity in a social network can be beneficial to a wide range of application domains, for example, political campaign, the promotion of health practices, marketing, and so on. Among all of these applications, a fundamental problem is to find the individuals in a social network with high structural diversity. Given a social network, we study a problem of finding top-k individuals with the highest structural diversity in this paper. Following the definition in [22] , the structural diversity of a node u is the number of connected components in a subgraph induced by u's immediate neighbors. Take the network in Figure 1 (a) as an example. The structural diversity of vertex f is 2, as the induced subgraph by f 's neighbors shown in Figure 1 (b) has two connected components.. To find the top-k vertices with the highest structural diversity, a naive method is to compute the structural diversity for all the vertices and then return the top-k vertices. Clearly, such a naive method is too expensive. To efficiently find the top-k vertices, the idea of traditional top-k query processing techniques [12] can be used, which finds the top-k answers according to some heuristic search order, and prunes the search space based on some upper bound score. Following this framework, in our problem, we have to address two key issues: (1) how to develop an effective upper bound for the structural diversity of a vertex, and (2) how to devise a heuristic search order in the computation.. In this paper, we propose several efficient and effective techniques to address these issues. We find that for two vertices connected by an edge, some structural information of them can be shared. For example, in Figure 1 (b) , vertex e forms a compo-nent of size 1 in f 's neighborhood. From this fact, we can infer that vertex f also forms a component of size 1 in e's neighborhood. Based on this important observation, the structural diversity computation for different vertices can also be possibly shared. To achieve this, we design a Union-Find-Isolate data structure to keep track of the known structural information of a vertex so as to avoid the computation of structural diversity for every vertex. A novel upper bound of the structural diversity is developed for pruning unpromising vertices effectively. Interestingly, the upper bound can be incrementally refined in the search process to become increasingly tighter. The main contributions are summarized as follows.. • We develop a novel Top-k-search framework to tackle the top-k structural diversity search problem. We design a UnionFind-Isolate data structure to keep track of the known structural information during the computation, and an effective upper bound for pruning.. • We devise a heuristic search order to traverse the components in a vertex's neighborhood. According to this search order, we propose a novel A * -search-based algorithm to compute the structural diversity of a vertex.. • We also design efficient techniques to handle frequent updates in dynamic networks and maintain the top-k results.. We use the Union-Find-Isolate structure and a spanning tree structure to efficiently handle edge insertions and deletions respectively.. • We conduct extensive experimental studies on large real networks to show the efficiency of our proposed methods.. The rest of this paper is organized as follows. We formulate the top-k structural diversity search problem in Section 2. We present a simple degree-based algorithm in Section 3. A novel Top-k-search framework is proposed in Section 4. We design two heuristic search strategies in Section 5 and discuss update in dynamic networks in Section 6. Extensive experimental results are reported in Section 7. We discuss related work in Section 8 and conclude this paper in Section 9. In this paper, we study the top-k structural diversity search problem motivated by a number of network analysis applications. We develop a novel Top-k-search framework to tackle this issue. Specifically, we design a Union-Find-Isolate data structure to keep track of the known structural information of each vertex, and an effective upper bound for pruning. We evaluate the proposed algorithms on 13 large networks, and the results demonstrate the effectiveness and efficiency of the proposed algorithms.. Our study in this paper serves as the first step to the exciting topic of top-k structural diversity search. [22] gives two more definitions of structural diversity based on k-core [5] and k-truss [23] . It would be interesting to extend the proposed techniques to these two definitions as a future work. To the best of our knowledge, top-k structural diversity search has not been studied before. In the following, we briefly review the existing work that are related to ours.. First, our work is closely related to the work on top-k query processing. The goal of top-k query processing is to find k objects with the highest rank based on some pre-defined ranking function. A commonly used framework for this problem is to examine the candidates in a heuristic order and prune the search space using an upper bound. After the seminal work by Fagin et al. [10, 11] , a large number of studies on top-k query processing have been done for different application scenarios, such as processing distributed preference queries [4] , keyword queries [17] , set similarity join queries [25] . Recently, many studies take the diversity into consideration in top-k query processing, in order to return diversified ranking results [26, 18, 1, 16, 2, 27] . A comprehensive survey of top-k query processing can be found in [12] . Second, our proposed techniques are related to the algorithms for the triangle listing problem, which is to find all triangles in a graph. Itai and Rodeh in [13] first proposed an O(m 1.5 ) algorithm for the triangle listing problem. In [15] , Latapy proved that the time complexity O(m 1.5 ) is optimal. Subsequently, Schank and Wagner [21, 20] proposed a simpler and particularly fast solution with the optimal complexity based on the vertex ordering and efficient lookup of the adjacency lists for neighborhood testing. Recently, Chu and Cheng [7] proposed an I/O-efficient algorithm for triangle listing in a massive graph, which cannot fit into the main memory. We conduct extensive performance studies to evaluate the algorithms proposed in this paper. All algorithms are implemented in C++ and all the experiments are conducted on the Linux operating system with 2.67GHz six-core CPU and 50GB main memory.. Comparison methods: To the best of our knowledge, we are the first to study top-k structural diversity search. In the literature, no algorithms have been proposed to address this problem yet. Thus, we compare our algorithms with the degree-based approach (Algorithm 1) which serves as a baseline. We evaluate four algorithms.. • Deg : The degree-based approach in Algorithm 1.. • Bou : Top-k-search equipped with bound-search (Algorithm 4) and θ = 1.. • FB : Top-k-search equipped with fast-bound-search (Algorithm 5) and θ = ( In our experiments, we find that θ = (. √ m which is close to 1 always yields a good performance in the Top-k-search framework. For FB and A * -B, their performances are not very sensitive to the value of θ as long as θ ∈ (1.001, 1.05) on all datasets. Due to the lack of space, we do not show the curves by varying θ, and simply set θ = ( n t ) 1 √ m for both FB and A * -B. Evaluation metrics: We use the running time and the number of vertices whose structural diversity scores are computed in the search process as two metrics. The latter evaluates the number of vertices that are pruned by the algorithm. Datasets: We use 13 publicly available real-world networks covering social, communication, collaboration networks, and webgraphs. The network statistics are shown in Table 1 . Except for Epinions, Digg and KDDTrack1 1 which are from their respective websites, the other 10 networks are downloaded from the Stanford Network Analysis Project (snap.stanford.edu). We treat all the networks as undirected.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Mixing doubly stochastic quantum channels with the completely depolarizing channel It is proved that every doubly stochastic quantum channel that is properly averaged with the completely depolarizing channel can be written as a convex combination of unitary channels. As a consequence, we find that the collection of channels expressible as convex combinations of unitary channels has non-zero Borel measure within the space of doubly stochastic channels. Let X and Y be finite-dimensional complex Hilbert spaces, and let L (X ) and L (Y ) denote the sets of linear operators acting on these spaces. By a super-operator one means a linear mapping of the form. Such a super-operator is said to be admissible if and only if it is both completely positive and preserves trace. In the usual (Schrödinger) picture of quantum information, the admissible superoperators represent valid physical operations, transforming a system having associated space X to one with associated space Y. For this reason, admissible super-operators are also commonly referred to as channels. A super-operator Φ is said to be unital if it is the case that Φ(1 X ) = 1 Y , for 1 X and 1 Y denoting the identity operators on X and Y, respectively. Super-operators that are both unital and admissible are said to be doubly stochastic. The existence of a doubly-stochastic channel of the form (1) obviously requires that X and Y have equal dimension d, which is assumed hereafter in this paper.. Among the doubly stochastic channels are the mixed-unitary 1 channels: those for which there exists a collection of unitary operators U 1 , . . . , U N ∈ U (X , Y ) and a probability distribution (p 1 , . . . , p N ) so that. 1 The term random unitary channel is commonly used. However, this term is easily confused with a different notion that is now common in quantum information theory-that of a fixed unitary operator that is randomly chosen according to some measure (frequently the Haar measure). For this reason, the term mixed-unitary channel is suggested in this paper.. for all X ∈ L (X ). (The notation U (X , Y ) refers to the set of unitary mappings from X to Y.). For the case d = 2 it holds that every doubly stochastic channel is mixed-unitary, but for larger d this is no longer the case: for all d ≥ 3 there exist doubly stochastic channels that are not mixedunitary [Tre86, KM87, LS93] . While it is computationally simple to check that a given channel is doubly stochastic, there is no efficient procedure known to check whether a given channel is mixed-unitary.. Several papers have studied properties of mixed-unitary channels within the last several years. For instance, Gregoratti and Werner [GW03] gave a characterization, with an interesting physical interpretation, of the class of mixed-unitary channels: they proved that a given channel is mixed-unitary if and only if it has full quantum corrected capacity, which assumes that the correction procedure is permitted arbitrary measurements on the channel's environment. Smolin, Verstraete, and Winter [SVW05] further investigated this notion of capacity, and conjectured that many copies of any doubly stochastic channel can be closely approximated by a mixed-unitary channel. Buscemi [Bus06] investigated bounds on the number N required in the expression (2) for different channels; Audenaert and Scheel [AS08] investigated conditions under which channels are mixed-unitary; and Mendl and Wolf [MW08] proved several facts about mixed-unitary channels, including the fact that a non-mixed-unitary channel can become mixed-unitary when tensored with the completely depolarizing channel. Rosgen [Ros08] recently proved that various additivity questions about general channels reduce to the same questions on the (potentially simpler) class of mixed-unitary channels.. This paper proves a simple fact about mixed-unitary channels, which is that every doubly stochastic channel becomes mixed-unitary when properly averaged with the completely depolarizing channel. To state this fact more precisely, let us write Ω : L (X ) → L (Y ) to denote the completely depolarizing channel, which is defined as. for every operator X ∈ L (X ). This channel is well-known to be mixed-unitary, as it is representable as a uniform mixture over the discrete Weyl operators (also known as the generalized Pauli operators). The claimed fact is now stated in the following theorem. As has been discussed by several authors, the Choi-Jamiołkowski representation reveals density operator analogues to various facts about channels. In the present case, the mapping. gives a linear bijection from the set of doubly stochastic channels Φ : L (X ) → L (Y ) to the collection of density operators ρ ∈ D (Y ⊗ X ) whose reduced states on both Y and X are completely mixed:. Corollary 2 therefore establishes that within the smallest affine subspace that contains all such density operators, there is a ball of positive radius around the maximally mixed state containing only states that are expressible as mixtures of maximally entangled states. When expressed in these terms, it is appropriate to compare the results of the previous section with an analogous fact concerning the set of separable states, which are those expressible as mixtures of unentangled states. Specifically, it follows from the results proved in [GB02] that any unit-trace Hermitian operator ρ ∈ L (Y ⊗ X ) satisfying. is a separable density operator. In contrast, Corollary 2 establishes that any unit-trace Hermitian operator ρ ∈ L (Y ⊗ X ) satisfying both the linear constraints (8) and the bound. is expressible as a convex combination of maximally entangled states. (It is clear that operators not satisfying the constraints (8) are not expressible as convex combinations of maximally entangled states, regardless of their distance from the maximally mixed state.) It is noted that the ball defined by (10) is properly contained in the one defined by (9). All of the states contained in the smaller ball are therefore representable at two extremes: as convex combinations of unentangled states and convex combinations of maximally entangled states. To what extent this fact can be generalized is an interesting question for future research.

O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Unifying Information-Theoretic Framework for Independent Component Analysis t e r r y 0 s a l k . e d u A b s t r a c t -w e show that different theories recently proposed for independent component analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pearlmutter and Parra [I] and Cardoso [2] showed that the infomax approach of Bell and Sejnowski [3] and the maximum likelihood estimation approach are equivalent. We show that negentropy maximization also has equivalent properties, and therefore, all three approaches yield the same learning rule for a fixed nonlinearity. Girolami and Fyfe [4] have shown that the nonlinear principal component analysis (PCA) algorithm of Karhunen and Joutsensalo [5] and Oja [6] can also be viewed from information-theoretic principles since it minimizes the sum of squares of the fourth-order marginal cumulants, and therefore, approximately minimizes the mutual information [7]. Lambert [8] has proposed different Bussgang cost functions for multichannel blind deconvolution. We show how the Bussgang property relates to the infomax principle. Finally, we discuss convergence and stability as well as future research issues in blind source separation. Recently, blind source separation by independent component analysis (ICA) has received attention because of its potential applications in signal processing such as in speech recognition systems, telecommunications, and medical signal processing. The goal of ICA is to recover independent sources given only sensor observations that are unknown linear mixtures of the unobserved independent source signals. In contrast to correlation-based transformations such as principal component analysis (PCA), ICA not only decorrelates the signals (2nd-order statistics) but also reduces higher-order statistical dependencies, attempting to make the signals as independent as possible.. Two different research communities have considered the analysis of independent components. On one hand, the study of separating mixed sources observed in an array of sensors has been a classical and difficult signal processing problem. The seminal work on blind source separation was by Herault and Jutten [9] where they introduced an adaptive algorithm in a simple feedback architecture that was able t o separate several unknown independent sources. Their approach has been further developed by Jutten and Herault [lo] , Karhunen and Joutsensalo [5] , and Cichocki et al. [Ill. Comon [7] elaborated the concept of independent component analysis and proposed cost functions related to the approximate minimization of mutual information between the sensors.. In parallel to blind source separation studies, unsupervised learning rules based on information theory were proposed by Linsker [12] . The goal was to maximize the mutual information between the inputs and outputs of a neural network. This approach is related to the principle of redundancy reduction suggested by Barlow [13] as a coding strategy in neurons. Each neuron should encode features that are as statistically independent as possible from other neurons over a natural ensemble of inputs; decorrelation as a strategy for visual processing was explored by Atick [14] . Nadal and Parga [15] showed that in the low-noise case, the maximum of the mutual information between the input and output of a neural network implied that the output distribution was factorial; that is, the multivariate probability density function (p.d.f.) can be factorized as a product of marginal p.d.f.s. Roth and Baram [16] and Bell and Sejnowski [3] independently derived stochastic gradient learning rules for this maximization and applied them, respectively, to forecasting, time series analysis, and the blind separation of sources. Bell and Sejnowski [3] put the blind source separation problem into an information-theoretic framework and demonstrated the separation and deconvolution of mixed sources. Their adaptive methods are more plausible from a neural processing perspective than the cumulant-based cost functions proposed by Comon [7] . A similar adaptive method for source separation was proposed by Cardoso and Laheld [17] .. Other algorithms for performing ICA have been proposed from different viewpoints. Maximum likelihood estimation (MLE) approaches to ICA were first proposed by Gaeta and La-I coume [18] and elaborated by Pearlmutter and Parra [I] . Girolami and Fyfe [19, 20] , motivated by information-theoretic indices for exploratory projection pursuit (EPP), used marginal negentropyl as a projection index and showed that kurtosis-seeking projection pursuit will extract one of the underlying sources from a linear mixture. A multiple output EPP network was developed to allow full separation of all the underlying sources [20] . Nonlinear PCA algorithms for ICA which have been developed by Karhunen be put into a unifying framework for the source separation problem based on an information theoretic approach. The original infomax learning rule for blind separation by Bell and Sejnowski [3] was suitable for super-Gaussian sources. Girolami and Fyfe [19] derive, by choosing negentropy as a projection pursuit index, a learning rule that is able to blindly separate mixed sub-and super-Gaussian source distributions. Lee, Girolami and Sejnowski [23] show that the learning rule is an extension of the infomax principle satisfying a general stability criterion and preserving the simple architecture of Bell and Sejnowski [3] . When optimized using the natural gradient [24] , or equivalently, the relative gradient [17], the learning rule gives superior convergence. Simulations and results on real-world physiological data show the power of the proposed methods [23] .. This paper is organized as follows. In Section 2, we formulate the problem and the assuinptions usually made in ICA. Section 3 reviews the infomax approach by Bell and Sejnowski [3] . Sections 4-8 describe, respectively, the relation between infomax, MLE, negentropy maximization, nonlinear PCA, higher-order statistics, and the Bussgang property. In Section 9, we discuss convergence properties and stability of the proposed algorithms. Potential applications and further research issues are discussed in Section 10. We have unified several lines of research on ICA within an information-theoretic framework and conjecture that this framework is well suited to further investigate ICA from many different theoretical viewpoints.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O