Fast detection of common geometric substructure in proteins We consider the problem of identifying common threedimensional substructures between proteins. Our method is based on comparing the shape of the a-carbon backbone structures of the proteins in order to find 3D rigid motions that bring portions of the geometric structures into correspondence. We propose a geometric representation of protein backbone chains that is compact yet allows for similarity measures that are robust against noise and outliers. We represent the structure of the backbone as a sequence of unit vectors, defined by each adjacent pair of a-carbons; we then define a measure of the similarity of two protein structures baaed on the RMS (root mean squared) distance between corresponding orientation vectors of the two proteins.Our measure has several advantages over measures that are commonly used for comparing protein shapes, such as the minimum RMS distance between the 3D positions of corresponding atoms in two proteins. This Permission to nlake digital or hard copies of all or part of this wrk for personal or classroom me is grmted without f'te prmidrd that copies are not made or distributed for prolit or cornmcrrial advantage and that copies bear this notice and the full citation on rhe first page. To copy otherwise, to republish. to post on servers or to redistribute to lists. requires prior specific permission and.:or a fee.'99 Lyon France Copyright ACM 1999 I-581 13-069-4/99/04...$5.00 104 new measure behaves well for identifying common substructures, in contrast with position-based measures where the nonmatching portions of the structure dominate the measure. At the same time, it avoids the quadratic space and computational difficulties associated with methods based on distance matrices and contact maps. We show applications of our approach to detecting common contiguous substructures in pairs of proteins, as well as the more difficult problem of identifying common protein domains (i.e., larger substructures that are not necessarily contiguous alorig the protein chain). As an increasing number of protein structures become known, the need for algorithms to analyze threedimensional conformational structure increases as well. The search for common substructure is of value in uncovering relationships among different proteins -for inferring similarities in function and discovering common evolutionary origins. There is now widespread agreement that similarities among distantly related proteins are often preserved at the level of three-dimensional structure, even after very little similarity remains at the sequence level [7] . W e are interested in developing efficient comparison methods for three-dimensional protein structures.. The goal of our work is to automatically identify common substructures shared by two proteins, based on similarity in their three-dimensional structure.. A key element of our approach is the development of a geometric notion of similarity that is appropriate to the task of protein structure comparison. This is a problem that has received considerable attention in the biological research literature (cf. [22] ), and a variety of different methods have been proposed.. For protein chains that are very similar (e.g., candidate structures produced in crystallography experiments), a common measure of similarity is the RMS (root mean square) distance between the positions of corresponding atoms of the proteins, after the proteins have been superimposed by a three-dimensional rigid motion (e.g. [lo, 191) . While this approach is reasonable for comparing two structures that are highly similar, it does not work well for finding similar substructures in otherwise dissimilar proteins. The difficulty with dissimilar proteins is that the portions of the structure that do not match dominate the RMS value, making it difficult to identify matching substructures. This is an instance of the classical issue of outliers, or bad data, dominating the measure in fitting or estimation problems.. Other methods for structural matching address the limitations of techiques that use the RMS distance. One widespread set of approaches is based on comparing the distance matrices or contact maps of two proteins (see e.g. [3, 9, 13, 24, 251) . The distance matrix records all pairwise distances between selected atoms within a given protein -often for all the a-carbons of the residues. For a protein with n residues, such a distance matrix therefore contains O(n2) entries. The distance matrix representation is invariant with respect to the position and orientation of the protein; thus the distance matrices for two proteins can be compared directly, without the need to find a rigid motion superimposing them as RMS distance approaches require. The use of distance matrices has a number of drawbacks, however. First of all, a full distance matrix is quadratic in the size of the protein it represents. Moreover, algorithms for aligning distance matrices have typically been computationally expensive; identifying corresponding substructures between two proteins requires finding corresponding submatrices that are nearly the same, and this appears to be a difficult optimization problem (e.g. [7, 91) .. Finally, a different set of approaches to structure comparison is based on using a purely local representation of a protein chain. Some common local representations are the set of (4, $J) torsion angles along the backbone [3, 223; the set of virtual-bond angles and virtualbond dihedral angles [ll, 12, 211 between consecutive a-carbons; and variations on these (e.g. (17, IS] ). Such representations are compact and independent of the position and orientation of the protein; however, there are settings in which two proteins with similar local representations can be geometrically very different. Consider, for example, two protein structures that are the same except for a bend in the middle. Although the sets of three-dimensional coordinates and pairwise distances of the proteins will be quite different, the data in a local representation of the two proteins will be the same except for those residues directly at the bend.. In this paper we present a new algorithm for efficiently detecting common substructures in proteins, based on a comparison methodology for protein chains that has both global and local aspects. For a given protein, we begin with the sequence of a-carbons that form its backbone chain. Rather than encoding the three dimensional coordinates of these atoms, we record the unit vector zli in the direction from the ith a-carbon to the (i + l)St a-carbon along the chain. The distance between successive a-carbons is essentially a fixed constant (approximately four angstroms), and thus this set of unit vectors captures the structure of the backbone. Indeed, these unit vectors correspond to the tirtual-bond vectors of the protein 111, 12, 211. By chaining the unit vectors vi end to end, we obtain the standard model of a protein as a sequence of a-carbons in space. By placing all of the unit vectors at the origin of R3, we can view the protein as a sequence of points on the unit sphere. We will represent the protein in this latter way: by its collection {vi} of unit vectors with respect to a common origin. Thus we see that our representation has both a local and a global nature: although each individual vector vi records the position of the (i + l)st a-carbon in a local reference frame centered at the ith a-carbon, our representation implicitly situates all the unit vectors {vi} in a common global reference frame. This is contrast to traditional approaches that have used virtual bonds among a-carbons to define purely local representations.. Our representation is therefore compact, having size O(n) for a protein with n residues, and it reflects global aspects of a protein's structure -introducing a single bend in the middle of structure affects aZE the vectors vi that come after the bend. Our unit-vector representation is not independent of the orientation of the protein, since the vectors {vi} are placed in a global frame of reference. However, we will see that unlike representations based on raw three-dimensional coordinates, the unit-vector representation allows for geometric similarity measures that are robust against outliers.. Our algorithm for comparing two protein chains, based on the unit-vector representation, consists of the following three steps.. Determine a small set of sh@s, or contiguous alignments, that are likely to include those alignments that bring matching substructures into 3D correspondence.. For each shift chosen above, determine the contiguous substructures for which there is a 3D geometric alignment (i.e., substructures of the two proteins that can be superimposed using a 3D rigid motion).. Combine the contiguous substructures chosen above to identify common protein domains (larger substructures that are not necessarily contiguous along the 'protein chain) that can be superimposed.. The first step of our algorithm runs in O(nlogn) time, where n is the number of a-carbons in the larger of the two proteins being compared. The second step takes 0(n) time for each shift identified in step one; typically only the best 10 shifts provide any useful information, but our current implementation checks the best 20. The running time of the third step depends on s, the number of substructures found in the second step.. Our approach makes use of two very different kinds of alignment. The first of these, employed in Step 1, is the alignment of two sequences by a simple shift relative to one another. We refer to this as contiguous alignment, as it is simpler than the standard biological notion of sequence alignment (which further allows for insertions and deletions into a sequence). More formally, for integers k = 0, 1, . . . , n -1, we match each residue i of the shorter protein with residue i + k of the longer protein. Here addition is taken modulo n, so that our shift includes a "wrap-around".. The wrap-around ensures that each unit vector in protein A is compared against each unit vector in protein B.. The second kind of alignment that we employ is geometric alignment, which is the 3D positioning of two proteins such that corresponding atoms in the two structures are approximately superimposed.. Our method will be based on a variant of the RMS distance, measuring differences between the global orientation vectors vi at corresponding a-carbons of two proteins, rather than measuring differences between the positions of the a-carbons themselves. The key underlying observation is that the difference between two orientation vectors is bounded by a small constant, whereas the distance between two points can grow arbitrarily large. This makes the unit-vector RMS (URMS) that we propose naturally robust with respect to differences between nonmatching parts of the structure, whereas the conventional position-RMS is not.. The RMS distance.. A basic method for determining geometric similarity between two sets of points is the root mean square (RMS) distance on point sets. This measure of distance requires a one-to-one correspondence between the point sets. The standard (positionbased) RMS distance between point sets A = {al, . . . , a,} and B = {br,. . . , bn} is defined as the square root of the sum of the squared distances between the points of A and the corresponding points of B. Generally one seeks the best possible RMS distance; that is, the RMS distance between A' and B where A' is the set A after application of the 3D rigid motion that minimizes the RMS distance. We refer to this as the minimum RMS distance between A and B.. To compute the minimum RMS distance, first transform A to make it have the same center-of-mass as B. Then compute the correlation matrix for A and B. This is a 3 x 3 matrix in which the (p, q) entry is ~~=r ai,pbi,,. This matrix is factored using the singular value decomposition; the resulting factors can be used to construct in constant time both (1) the orthogonal matrix that brings A into the closest possible match with B and (2) the resulting RMS distance. For a concise description of the algorithm and an explanation of why it works see Golub and Van Loan 161 (where it is referred to as the orthogonal Procrustes problem) or Lesk [lo] .. The minimum RMS distance between two sets of atoms in R3 is useful primarily when aligning two structures that are extremely similar. Thus, it is widely used for aligning structures produced in the course of crystallographic or NMR analysis of molecules.. It is less appropriate for comparing structures that have only a distant similarity. This is due to its sensitivity to outliers. Indeed, a few outliers can have a significant effect on the resulting minimum RMS distance. For instance, consider sets A and B, each with n points, where the minimum RMS distance between A and B is 0. Now suppose we take the point bl and gradually move it along a straight line. As we move bl, the minimum RMS distance between A and B increases without bound, and the rigid motion corresponding to this minimum distance also changes so that the congruent point sets A -{ur } and B -{ bl} become arbitrarily far apart. Thus a small number of gross outliers can hide a large shared substructure.. Another issue with the minimum RMS distance is that points far from the center of mass are, in effect, weighted more heavily than points near the center of mass. Consequently, a common "core" shared by two structures is difficult to detect using the minimum RMS distance. This is problematic in the case of proteins, where some crucial similarities occur near the center of mass.. Our Approach.. Using an RMS distance on the unitvector representation {vi} allows us to avoid the drawbacks discussed above while retaining the computational efficiency of the minimum RMS calculations. Recall our view of the representation {vi} as consisting of a sequence of points on the unit sphere in R3. To compare two proteins, with sets of unit vectors {vi} and {wi}, we compute the minimum RMS distance between the two sets {vi} and {wi}, viewed as subsets of the unit sphere. Since we wish the origins of the vectors to remain fixed in space, we minimize the RMS distance only over possible rotations of the vectors, not over translations. Henceforth we use the term position-RMS to refer to the standard, position-based, minimum RMS distance; we use the term unit-vector RMS (URMS) to refer to the minimum RMS distance for a pair of sets in our unit-vector representation.. Because the maximum squared distance between any two unit vectors is 4, no single outlier can have a large effect on the URMS. As a result, the URMS exhibits a natural resilience to the presence of outliers, such as those arising due to portions of two protein structures that do not match.. Other Related Work.. Above, we have indicated a number of the approaches that have been proposed for protein structure comparison. Here, we discuss several of these in more detail; see [2, 8, 221 for recent surveys on the topic.. There has been work on discovering geometric alignments that are not required to respect the order of the sequence, using distance matrices on o-carbons and secondary structural elements of proteins (Vriend and Sander [24] , Holm and Sander [9] , Grindley et al. [5] ). In particular, Vreind and Sander [24] use the distance matrix to find pairs of matching fragments of two proteins and then combine them into larger units which are compatible in R3. The latter step is based on comparisons of the centers of mass of the fragments and of relative rotations between combined fragments. Yee and Dill [25] use distance matrices as part of an algorithm whose high-level structure is similar to ours. They also search over contiguous alignments of two protein sequences to discover similarities, but their notion of similarity is different from ours. It is based on shifting the alignments of the distance matrices of the two proteins and computing the "distance" between the matrices (by means of the Frobenius norm). The shifts that give the smallest norms are predicted to yield matching contiguous alignments. Though the running time is not discussed explicitly in their work, it is easy to see that it can be O(n3). (It is possible that an application of the Fast Fourier Transform could improve this to O(n2 log n).) There has also been work by a number of groups on extending classical sequence alignment algorithms to handle pairwise relationships among residue positions (Taylor and Orengo [23] , Orengo et al. [14] , Sali and Blundell [20] ).. Rackovsky and Scheraga [17, 181 develop a purely local representation of protein chains, by defining discrete analogues of curvature and torsion in terms of the virtual bond vectors. They propose a method of searching for structural similarities based on comparing these parameters for different proteins [16, 181. Fischer et al [4] apply geometric hashing to find matching pairs of atoms (alpha carbons) between two molecules. The common subsequences they find are not necessarily consecutive. To use geometric hashing, a base molecule is preprocessed to find all possible matching coordinate frames. This may involve up to O(n3) pre-processing and storage, since a rigid motion in three dimensions is specified by its action on three points. (Distance constraints imposed by the protein can improve this bound somewhat.) Given a second target molecule, each triplet of points "votes" for a rigid motion with respect to the base molecule; a large number of common votes indicates a possible large match between the two molecules. Assuming that the rigid motions are indexed in a welldistributed hash table, the running time is O(n3). Recent work on this technique has suggested that it may be possible to reduce the running time for protein matching applications to a roughly quadratic bound [15] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Optimal approximation of elliptic problems by linear and nonlinear mappings III: Frames We study the optimal approximation of the solution of an operator equation A(u) = f by certain n-term approximations with respect to specific classes of frames. We consider worst case errors, where f is an element of the unit ball of a Sobolev or Besov space B t q (L p (Ω)) and Ω ⊂ R d is a bounded Lipschitz domain; the error is always measured in the H s -norm. We study the order of convergence of the corresponding nonlinear frame widths and compare it with several other approximation schemes. Our main result is that the approximation order is the same as for the nonlinear widths associated with Riesz bases, the Gelfand widths, and the manifold widths. This order is better than the order of the linear widths iff p < 2. The main advantage of frames compared to Riesz bases, which were studied in our earlier papers, is the fact that we can now handle arbitrary bounded Lipschitz domains-also for the upper bounds. We study the optimal approximation of the solution of an operator equation (1) A(u) = f, where A is a linear operator (2) A : H → G from a Hilbert space H to another Hilbert space G. We always assume that A is boundedly invertible, hence (1) has a unique solution for any f ∈ G. We have in mind the more specific situation of an operator equation which is given as follows.. Assume that Ω ⊂ R d is a bounded Lipschitz domain and assume that. is an isomorphism, where s > 0. For the exact definitions of Lipschitz domains and spaces of distributions defined on such domains we refer to the Appendix, see also [9] . Now we put H = H s 0 (Ω) and G = H −s (Ω). Since A is boundedly invertible, the inverse mapping S : G → H is well defined. This mapping is sometimes called the solution operator-in particular if we want to compute the solution u = S(f ) from the given right-hand side A(u) = f . We study different mappings S n for the approximation of the solution u = A −1 (f ) for f contained in F ⊂ G. We consider the worst case error (4) e(S n , F, H) = sup. where F is a normed (or quasi-normed) space, F ⊂ G. In our main results, F will be a Sobolev or Besov space. 1 Hence we use the following commutative diagram. Here I : F → G denotes the identity and S F the restriction of S to F . Then one is interested in approximations that have an optimal order of convergence depending (L p (Ω)), which hold for 1 ≤ p ≤ ∞, t ≥ s, see [45] , our results are valid also for Sobolev spaces.. on n, where n denotes the degrees of freedom. For our purposes, the following approximation schemes are important. Consider the class L n of all continuous linear mappings S n : F → H,. with arbitraryh i ∈ H. The worst case error of optimal linear mappings is given by the approximation numbers or linear widths e lin n (S, F, H) = inf Sn∈Ln e(S n , F, H).. We may also use nonlinear approximations with respect to a Riesz basis R of H, i.e., we consider the class N n (R) of all (linear or nonlinear) mappings of the form. where the c k and the i k depend in an arbitrary way on f . Then the nonlinear widths e e(S n , F, H).. Here R C denotes a set of Riesz bases for H where C indicates the stability of the basis, i.e., we require B/A ≤ C and A, B are the Riesz constants of the basis. The investigation of these widths e non n,C and its comparison with the linear widths have been the major part of our analysis in [8, 9] . This has continued earlier research on related topics, cf. e.g. [24, 38, 39, 40] . The next type of widths we are interested in has served as a very useful tool in our analysis of the widths e non n,C in [9] . The manifold widths are related to the class C n of continuous mappings, given by arbitrary continuous mappings N n : F → R n and ϕ n : R n → H. Again we define the worst case error of optimal continuous mappings by where S n = ϕ n • N n . These numbers have been studied in [13, 27] and later in [9, 14, 16, 17] . As mentioned above we have studied the relationships of these widths in [9] . It has turned out that for problems as in (3) Hence, if p < 2 then there is an essential difference in the behavior, nonlinear approximations can do better than linear ones. This paper is a continuation of [8, 9] . We are again interested in optimal nonlinear approximation schemes, but this time not related to Riesz bases but to classes of frames. The motivation for this is given by the following observations. In [9] , we presented upper and lower bounds for e non n,C (S, F, H). The proof of the lower bound was quite general and used the fact that e non n,C (S, F, H) can be estimated from below by the manifold widths e cont n (S, F, H) up to some constants. In contrary to this, the proof of the upper bound was based on norm equivalences of Besov norms with weighted sequence norms that are induced by a biorthogonal wavelet basis. However, this restricts the choice of the underlying domain Ω ⊂ R d since on a general Lipschitz domain the construction of a suitable wavelet basis might be very complicated or even impossible. This problem becomes less serious in the frame setting since a suitable wavelet frame always exists, see Section 5.2 for a detailed discussion. Moreover, in recent years the application of frame methods for the numerical resolution of the solution u in (1) has become a field of increasing importance. Especially, it has been possible to derive adaptive wavelet frame schemes that are guaranteed to converge for a wide range of problems [6, 7, 37] . Therefore it is important to clarify the power that frame schemes can have, in principle.. In this paper, we give a first answer. Our main result states that the nonlinear frame widths show the same asymptotic behavior as the e non n,C (S, F, H), where we now can allow arbitrary bounded Lipschitz domains.. There is an interesting difference to the Riesz bases case. In the frame setting, we do not work with arbitrary n-term approximations, but only with those induced by a frame pair, see Section 2.2 for details. The reason is that, for practical applications, only these canonical representations are used. Actually we prove that if we would allow arbitrary n-term approximations then the associated frame widths would be zero. Moreover, certain conditions related to stability must be satisfied by the admissible frames. Fortunately, these conditions are always satisfied for the known constructions of wavelet frames on Lipschitz domains. This paper is organized as follows. In Section 2, we describe the basic setting. First of all, we introduce and discuss the frame concept as far as it is needed for our purposes. Then, in Subsection 2.2, we define the nonlinear frame widths and prove some basic properties that are needed in the sequel. Section 3 contains the main results of this paper. In the next section two examples are discussed: the Poisson equation for Lipschitz domains and a Fredholm integral equation of the first kind (the single layer potential). Proofs of our main results are given in Section 5. For general Hilbert spaces H and G we show that similar to the Riesz bases case the nonlinear frame widths can be estimated from below by the manifold widths. Then, for the more specific case of Besov spaces on Lipschitz domains, we also prove an upper estimate which shows that the asymptotic behavior is the same as for the Riesz basis case-but this time for arbitrary bounded Lipschitz domains.. Notation. We write a ≍ b if there exists a constant c > 0 (independent of the context dependent relevant parameters) such that. One-sided estimates of this type are denoted by a < ∼ b. All unimportant constants will be denoted by c, sometimes with additional indices. Identity operators are always denoted by I, also sometimes with additional indices.

O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The augmented knowledge workshop The story of my involvement with on-line workstations begins in early 1951, with a vision and a life-time professional commitment. Over thirty-four years of pursuit have created a lot of personal history, and the object of this historical exercise, the workstation, occupies a unique place in it.. For me, a workstation is the portal into a person's "Augmented Knowledge Workshop" --the place in which he finds the data and tools with which he does his knowledge work, and through which he collaborates with similarly equipped workers. And further, I consider that the large system of concepts, skills, knowledge, methods, etc. on the human side of the workstation has to be taken into account, in a balanced way, when pursuing increased human effectiveness. So, my workstation-history story embraces a rather large sphere.. The task of writing an historical piece is unfamiliar enough to cause me difficulty by itself, but the associated stirring of old records and old memories has added near overwhelming burden --dreams, events, people, stresses, pleasures, disappointments, the firsts and the failures. Now, what from all of this --and how to organize it --will make an appropriate "history" paper? I could provide a solid measure of objective reporting --events and dates, etc. Regarding the general environment that is relevant to the workstation topic, I have been an involved observer of related computer history since 1951. I watched and experienced the supportive hardware, languages and architecture evolve, witnessed the people and efforts that brought timesharing into being, and was even more closely involved with the emergence of computer networks. Through all of this, I was wholly focused on what these things could do for people at workstations. And then there was office automation and personal computers: you don't have to be an old guy to have watched these emerge, but I'm sure they looked different to me than to most.. I could also provide lots of objective reporting about the events and dates associated with the things I have caused or had a direct hand in. There seems to be a lot there that is quite relevant to this "history of the workstation" theme. It was dusty, laborious work, this process of brainstorming for candidates, culling and ordering and trying to describe them in some reasonable sequence and context.. But what I came to realize is that there is one, clearly dominant factor that underlies essentially every cause for any uniqueness that I might list for historical record. It isn't a technology, it isn't a science, and it isn't a marketing or business model. And I am going to give it dominant coverage in this paper.. As explained below, it is what I call my "Framework." It is based upon an intuitive conviction, emplanted in my head (apparently permanently) over thirty years ago, that the gains in human knowledge-work capability which we will achieve by properly harnessing this new technology will be very large. Metaphorically, I see the augmented organization or institution of the future as changing, not as an organism merely to be a bigger and faster snail, but to achieve such new levels of sensory capability, speed, power and coordination as to become a new species --a cat.. Based upon this conviction about huge potential gains for mankind, my Framework explains for me generally where such gains are going to come from, and provides strategic principles that can help guide a conscious pursuit of these gains.

O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Runtime vs. Manual Data Distribution for Architecture-Agnostic Shared-Memory Programming Models This paper compares data distribution methodologies for scaling the performance of OpenMP on NUMA architectures. We investigate the performance of automatic page placement algorithms implemented in the operating system, runtime algorithms based on dynamic page migration, runtime algorithms based on loop scheduling transformations and manual data distribution. These techniques present the programmer with trade-offs between performance and programming effort. Automatic page placement algorithms are transparent to the programmer, but may compromise memory access locality. Dynamic page migration algorithms are also transparent, but require careful engineering and tuned implementations to be effective. Manual data distribution requires substantial programming effort and architecture-specific extensions to the API, but may localize memory accesses in a nearly optimal manner. Loop scheduling transformations may or may not require intervention from the programmer, but conform better to an architecture-agnostic programming paradigm like OpenMP. We identify the conditions under which runtime data distribution algorithms can optimize memory access locality in OpenMP. We also present two novel runtime data distribution techniques, one based on memory access traces and another based on affinity scheduling of parallel loops. These techniques can be used to effectively replace manual data distribution in regular applications. The results provide a proof of concept that it is possible to scale a portable shared-memory programming model up to more than 100 processors, without modifying the API and without exposing architectural details to the programmer. We conducted a detailed evaluation of several data distribution methods for improving the scalability of OpenMP on NUMA multiprocessors. The simplest solution is to use the existing automatic page placement algorithms of the operating system, which include some NUMAaware algorithms such as round-robin and first-touch. The experiments have shown that this solution is not adequate. At the moderate scale of 16 processors, automatic page placement algorithms already perform worse than manual data distribution, whereas at larger scales, the performance of automatic page placement algorithms shows diminishing returns. The second step to attack the problem, i.e., introducing data distribution directives in OpenMP, reaches performance levels which verify the common belief that a data-parallel programming style is likely to solve the problem of memory access locality on NUMA architectures. Nevertheless, the experiments show that manual data distribution is not a panacea. The same or better performance can be obtained from carefully engineered runtime data distribution algorithms, based on dynamic page migration and affinity scheduling.. Dynamic page migration is effective under certain constrains regarding the granularity, regularity and periodicity of the memory access pattern. These constraints are important enough to justify further investigation, since several parallel codes do not possess all these properties. Our experiments with LU, a program with an aperiodic but fairly simple access pattern, gave us a hint that loop schedules with affinity links maintained across invocations of the same loop are able to localize memory accesses very well. We believe that several codes might benefit from flexible affinity scheduling strategies that move computation close to data, rather than data close to computation. It is a matter of further investigation and experimentation to verify this assumption.. The problem of granularity is hard to overcome because the overhead of page migration remains high. [The cost for migrating a page on the Origin2000 is approximately 1 ms.] It may worth the effort to investigate alternatives such as virtually addressed cache organizations that move the TLB out of the processor core and down in the memory hierarchy. Such organizations can greatly reduce the TLB consistency overhead, which dominates the page migration time. A software alternative is to parallelize the page migration procedure. The idea is to have each processor poll the access counters of locally mapped pages and forward pages that satisfy the migration criterion to the receiver. We expect that careful inlining of the page migration algorithm in OpenMP threads will reduce the interference between the page migration engine and the program and hopefully provide more opportunities for page migration under tight time constraints. At the very end, this paper proposed an alternative methodology based on dynamic compilation, which removes the overhead of runtime data distribution out of the critical path.. Irregular codes are a challenging domain for future work. Adaptive codes for example, have memory access patterns which are a-priori unknown and change at runtime in an unpredictable manner.. (39) Irregular parallel applications appear to be one class of programs where page migration is not an option and domain-specific knowledge may be required to encode proper algorithms for data and load distribution. The idea of introducing data distribution directives in shared-memory programming paradigms originates in the earlier work on data-parallel programming languages. (24, 27) Data-parallel languages use array distributions as the starting point of parallelization. Arrays are distributed along one or more dimensions, in order to let processors perform the bulk of the computation on local data and communicate rarely in a loosely synchronous manner. The communication is handled by the compiler, which has complete knowledge of the data owned by each processor. The computation is executed with the owner-computes rule, so that each processor computes using data available in local memory.. In order to use a data-parallel programming model on a NUMA multiprocessor, data distribution directives must be translated to distributions of the virtual memory pages that contain the array elements assigned to each processor. Communication is carried out directly through loads and stores in shared memory. From the performance perspective, the data-parallel programming style is attractive for NUMA architectures, because it establishes explicit bindings between computation and data. From the programming perspective though, data distribution compromises the simplicity of the shared-memory programming paradigm by exposing the data placement to the programmer. The implication is that the programming effort required to place data in memory and minimize the communication overhead might very well be as much as the effort required to obtain a scalable message-passing version of the program.. Several researchers and vendors have implemented data-parallel extensions to OpenMP. (13, 16, 17, 19) These extensions serve two purposes. The first is the distribution of data at page-level or element-level granularity. At page-level granularity, the unit of data distribution is a virtual memory page. At element-level granularity, the unit of data distribution is an individual array element. Since an element-level array distribution may assign two or more elements originally placed in the same page to different processors, the compiler may need to reshape arrays and rewrite subscripts to implement the distribution. The second purpose of data-parallel extensions to OpenMP is to explicitly assign loop iterations to specific memory modules to exploit locality. The objective is to assign each iteration to the memory module that contains all, or at least a good fraction of the data accessed by the loop body during that iteration. Data-parallel extensions of OpenMP have demonstrated significant performance improvements in simple numerical kernels like LU and SOR, albeit by modifying the native API and sacrificing portability. (13, 17, 19) We have developed a runtime system which applies dynamic locality optimizations to OpenMP programs. (21, 28) The runtime system records the number of accesses from each processor to each page in memory and applies competitive page migration algorithms at specific points of execution, where the access counters reflect accurately the memory access pattern of the program. In regular, iterative parallel codes, these points are the ends of the outer iterations of the loop that encloses the parallel computation. This technique is both accurate and effective in relocating early at runtime the pages that concentrate many remote accesses. In most cases, the runtime system stabilizes the placement of pages after the execution of the first iteration and this placement is optimal with respect to the access pattern, in the sense that each page is placed in the node that minimizes the latency of accesses to the page. (21) The algorithm may actually perform better than manual data distribution, because it can remedy cases in which manual data distribution places pages in a non-optimal manner. This can happen for example if there are no appropriate distributions for certain memory access patterns that exist in the program, or if manual distribution places a falsely shared page with data assigned to multiple nodes in a node that does not access the pages more frequently than the other sharers.. The results obtained so far show that carefully engineered page migration algorithms can render OpenMP programs immune to the page placement algorithm of the operating system. (20) This means that programs perform always as good as they do with the best-performing automatic page placement algorithm, no matter how their data is initially placed in memory. Unfortunately, there are also cases in which manual data distribution outperforms the best automatic page placement algorithms by sizeable margins. It remains to be seen if a dynamic page migration engine can provide the same advantage over automatic page placement algorithms in such cases. A second important restriction of page migration is that it can only implement block data distributions and under certain constraints. Implementing arbitrary single-dimensional or multi-dimensional distributions at runtime requires additional support from the compiler and the runtime system.

O O O B I O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Feature selection on movement imagery discrimination and attention detection Abstract Noninvasive brain-computer interfaces (BCI) translate subject's electroencephalogram (EEG) features into device commands. Large feature sets should be downselected for efficient feature translation. This work proposes two different feature down-selection algorithms for BCI: (a) a sequential forward selection; and (b) an acrossgroup variance. Power rar ratios (PRs) were extracted from the EEG data for movement imagery discrimination. Event-related potentials (ERPs) were employed in the discrimination of cue-evoked responses. While center-out arrows, commonly used in calibration sessions, cued the subjects in the first experiment (for both PR and ERP analyses), less stimulating arrows that were centered in the visual field were employed in the second experiment (for ERP analysis). The proposed algorithms outperformed other three popular feature selection algorithms in movement imagery discrimination. In the first experiment, both algorithms achieved classification errors as low as 12.5% reducing the feature set dimensionality by more than 90%. The classification accuracy of ERPs dropped in the second experiment since centered cues reduced the amplitude of cue-evoked ERPs. The two proposed algorithms effectively reduced feature dimensionality while increasing movement imagery discrimination and detected cue-evoked ERPs that reflect subject attention. Current brain-computer interfaces (BCI) determine the intent of the users from a variety of electroencephalographic features [32] . BCIs enable the physically disabled as a consequence of neuromuscular disorders, amyotrophic lateral sclerosis, brainstem stroke or spinal cord injuries to control a device with their brain signals [32] . Noninvasive BCI applications commonly use scalp electroencephalogram (EEG) for laboratory and clinical applications. BCI systems usually adopt one of three operation types: in operant conditioning, the subject is extensively trained to control his own rhythms [33] ; in pattern recognition, a classification algorithm discriminates mental task performance by identifying the subject-specific EEG patterns corresponding to each mental task [26] ; and in eventrelated potential (ERP) detection, learning algorithms are employed to detect stimulus-evoked responses in a singletrial basis [13, 21] . Particularly when the latter two BCI approaches are adopted, as increasing number of features are used to train the classifier, the risk of over-fitting to the data increases. The effects of a low ratio of the number of samples to the number of features have been extensively discussed as the ''curse of dimensionality'' [7] . Since long training experiments are not practical, a dimensionality reduction technique should be employed to find a feature subset that minimizes the cross-validation error of mental task discrimination.. Two dimensionality reduction methodologies have been adopted in BCI research. In one, studies make use of common-spatial patterns (CSPs) [22, 29] , principal component analysis (PCA) [16, 25] , and independent component analysis (ICA) [17] , among others, that transform the original feature space into lower dimensional spaces. An alternative methodology is feature down-selection which produces a subset of the original features that are most relevant to discriminate subject performance. The greatest advantage of the latter methodology is the effective reduction of BCI computational complexity. Among the methods proposed in previous studies to down-select feature sets are wrapper or filter methods based on dependence on a learning technique [34] . Wrapper methods typically use the predictive accuracy or other performance measures of a pre-selected classifier to evaluate a feature subset. Some exemplars are recursive feature elimination [20] and other sequential selection algorithms [8, 19] . The filter methods separate feature selection from classifier training and produce feature subsets independent of the classifier. The relief algorithm [24] and PCA [2] are often used as filter methods. Genetic algorithms (GAs) are also popular in BCI research [9] .. This work proposes two different algorithms to downselect features: (a) a wrapper-type sequential forward selection (SFS) algorithm that adds features to the subset sequentially for task discrimination and (b) a filter type across-group variance (AGV) algorithm, based on a formulation of PCA that accommodates the group structure of the data set. Two different EEG feature types were used: power ratios (PRs) and event-related potentials (ERPs). While PR features were extracted for movement imagery discrimination, the ERP features were extracted in order to detect cue-evoked potentials [13, 21, 30] . The developed algorithms were applied to the data of five subjects with no previous BCI experience. The SFS and AGV algorithms are intended to improve feature selection by maximizing the classification accuracy of the learning algorithm and minimizing both the number of EEG channels selected and the computation time. The performances of the proposed algorithms in movement imagery discrimination were evaluated in comparison to three other feature selection algorithms in common use: recursive feature elimination (RFE) [20] , genetic algorithm (GA) [29] , and relief [24] .. The selection of ERPs intends to determine the optimal EEG channels to detect subject attention in two different experimental conditions: in Experiment 1, the subject was stimulated with center-out asymmetric arrows that are typically used for BCI calibration [3, 12, 26] ; in Experiment 2, the subject was stimulated with symmetric arrows that were balanced in the subject visual field. This study introduces two novel algorithms to down-select features for BCI applications. The proposed feature selection algorithms were compared with three other popular algorithms to select relevant PRs for discrimination of left versus right hand movement imagery performance. The AGV algorithm performed the best among all the other algorithms that were tested. AGV achieved classification errors between 12.5% and 38.1% with feature dimensionality reductions of more than 95%. Although SFS and RFE algorithms achieved similar classification accuracies, the former selected smaller feature subsets. The GA and relief algorithms ranked next in increasing error. Besides the best classification accuracy, the AGV algorithm ran six times faster than the SFS algorithm and eight times faster than the RFE algorithm.. As a wrapper-type algorithm, iterative selection has been frequently used for sequential feature selection in BCI research with promising results [8, 19] . Although the running time of the classifier is multiplied by a factor of m 2 (m being the number of features), algorithms such as SFS and RFE are becoming more practical due to the increasing computational power of laboratory grade computers. Additionally, such algorithms are straightforward to implement. The SFS algorithm selects features based on the Wilks' statistic which uses the eigenvalues of the transformation matrix calculated by an LDA classifier. Instead of assessing a confidence value of the F-statistic as the standard stepwise stop criterion [7] , the number of features to include in the classification model was optimized in a cross-validation loop. Therefore, this parameter could be customized for each subject based on the classification error. The applicability of the SFS algorithm to more than two data groups is straightforward since it relies on the LDA classifier ability to generalize for any number of groups [10] .. The proposed AGV algorithm is based on a formulation of PCA with supervised learning. PCA has been widely used as a dimension reduction technique in BCI research [25] . The proposed AGV algorithm uses principal components to down-select original features thus reducing computational complexity. Features are ranked according to their AGV in a truncated component space. Ranking algorithms usually order features according to a relevance Fig. 7 Event-related lateralization for left versus right arrows calculated for the channel pairs C4-C3, P8-P7, and O2-O1 from Subject A data in Experiment 1 (a) and Experiment 2 (b). Data have been previously re-referenced to common-average criterion but ignore the context of other features [14] . However, in the AGV algorithm, feature ranking implicitly considers the context of other features since a principal component is a linear combination of all features. As in the SFS algorithm, the number of features selected was optimized on a subject-basis in a cross-validation loop. The AGV score directly generalizes for more than two groups since the between group covariance matrix can be equally calculated for any number of groups.. The comparison of the two introduced methods reveals distinct results. The Wilks' test of significance revealed that discriminations calculated through the SFS algorithm were more significant than the AGV algorithm's. However, the AGV achieved lower cross-validation errors than SFS. As these results suggest, although the feature subsets computed through the SFS algorithm represented best the data used to train the classifier, they did not describe previously unseen data as well as the features calculated by the AGV algorithm. Moreover, in this study, the SFS algorithm selected more features than the AGV algorithm. On the one hand, smaller feature subsets contribute to less complex classification models which may increase model adaptability to new data. On the other hand, larger feature subsets retain more information about subject-specific EEG patterns and can benefit from low intertrial variability on the task-related responses.. In movement imagery discrimination, the frequency ranges 8-12 and 10-14 Hz on the C3 and C4 EEG channels were frequently selected by both proposed algorithms which reflects the reactivity of the ''rolandic mu rhythm'' to movement imagery tasks [32] . The discrimination relevance of some posterior channels (e.g., parietal and occipital locations) has also been reported earlier and appears related to the concurrent visual input of instructing cues [1] .. Event-related potentials in response to left versus right movement cues were also discriminated in a trial basis. The proposed algorithms achieved classification errors between 14% and 33% with feature dimensionality reductions of more than 90%, when symmetric cues were employed (Experiment 1). While channels at parietal (e.g., P7 and P8) and occipital (e.g., O1 and O2) sites were selected frequently among subjects in Experiment 1, such channels were barely selected or with less specificity in Experiment 2. The investigated ERL waveforms demonstrated a contra-lateral component between 150 and 350 ms poststimulus, prominent at parietal and occipital sites. Our results demonstrate that this component was modulated by visual stimuli parameters since it demonstrated higher amplitudes in Experiment 1 than in Experiment 2 [15] , and demonstrated higher latency in Experiment 2. Similar parietal and occipital ERPs have been associated with brain mechanisms dependent on visual spatial attention [21, 31] and/or movement intention [13, 30] . Although the amplitudes of such ERPs were largely attenuated by the presentation of balanced arrows, which suggests the manifestation of a visual spatial mechanism, the residual direction-congruent potentials on parietal and occipital channels (Fig. 7b ) might also unveil a movement intention mechanism [30] . The detection of these cue-evoked potentials was most prominent on P7, P8, O1, and O2 channels and confirms that the subject is indeed fixating and giving attention to the BCI paradigm. Therefore, this physiology might be used as a ''gate'' mechanism of the classification algorithm to validate the detection of a motor imagery event.. In conclusion, the proposed feature selection algorithms demonstrated their value for discrimination of both movement imagery tasks and cue-evoked responses by increasing classification accuracy, reducing the number of required EEG channels, and reducing computation times. In addition, use of sensory evoked potentials to detect fixation and attention, and the required causal time lags to motor intention, offer creative prospects to improve BCI [13] . Results from Subject A were presented for the three symmetric channel pairs C4-C3, P8-P7, and O2-O1 from both experiments z ¼ Ya:. Considering that z is the discrimination function and Y is the feature matrix. Initially, the SVD of the within-group covariance matrix W is calculated as: Five healthy human subjects, 25-32 years old, four males and one female, none of them under any medication, consented to participate in this study. The experiments were conducted under Institutional Review Board (IRB) approval at Penn State University.

B I O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Model-Based Tomographic Reconstruction of Objects Containing Known Components Abstract-The likelihood of finding manufactured components (surgical tools, implants, etc.) within a tomographic field-of-view has been steadily increasing. One reason is the aging population and proliferation of prosthetic devices, such that more people undergoing diagnostic imaging have existing implants, particularly hip and knee implants. Another reason is that use of intraoperative imaging (e.g., cone-beam CT) for surgical guidance is increasing, wherein surgical tools and devices such as screws and plates are placed within or near to the target anatomy. When these components contain metal, the reconstructed volumes are likely to contain severe artifacts that adversely affect the image quality in tissues both near and far from the component. Because physical models of such components exist, there is a unique opportunity to integrate this knowledge into the reconstruction algorithm to reduce these artifacts. We present a model-based penalized-likelihood estimation approach that explicitly incorporates known information about component geometry and composition. The approach uses an alternating maximization method that jointly estimates the anatomy and the position and pose of each of the known components. We demonstrate that the proposed method can produce nearly artifact-free images even near the boundary of a metal implant in simulated vertebral pedicle screw reconstructions and even under conditions of substantial photon starvation. The simultaneous estimation of device pose also provides quantitative information on device placement that could be valuable to quality assurance and verification of treatment delivery. This paper reports a novel tomographic reconstruction approach referred to as KCR that allows for incorporation of known components into the object model. The position and pose of such components are estimated simultaneously with the background attenuation. Application of this technique was demonstrated in images containing metallic pedicle screws within a spine surgery context, but the technique is general for any kind of component for which the shape and composition is known-including objects with heterogeneous composition. In simulated pedicle screw reconstructions, KCR largely eliminated artifacts associated with photon starvation and allowed clear visualization of anatomy right up to the boundary of the implant, whereas traditional FBP and PL approaches exhibited substantial artifacts. As such, we anticipate the KCR method to hold significant promise in cone-beam CT (CBCT)-based imaging for interventional guidance-e.g., in spine procedures [36] as well as a host of other interventions for which the introduction of a surgical device confounds 3-D image quality.. Moreover, KCR may be an important tool in dose reduction, since it allows for successful operation in a photon starvation regime. Investigation in the context of other applications, such as diagnostic CT, imaging of hip and knee implants, and CT-based biopsy needle guidance will be considered in future work.. We note that only a basic FBP reconstruction was used for comparison and that more sophisticated approaches that include strategies for metal artifact reduction would likely have a less dramatic difference between FBP and KCR. While it is outside of the scope of this paper, future investigations should compare such FBP-based artifact reduction methods [8] , [9] , [14] with KCR to determine these differences.. The value of incorporation of this kind of prior knowledge has previously been reported with the development of a related reconstruction approach, object-constrained CT (OCCT), in [26] [27] [28] . Similar observations were made that artifacts associated with high-density known components could be largely mitigated through the incorporation of prior knowledge. Both KCR and OCCT rely on Poisson objectives and a likelihood-based objective function. However, KCR adopts a penalty term that allows for the advantages of a penalized-likelihood approach, namely control of the noise-resolution tradeoff and improved convergence rates as well as improved fidelity of registration estimates (as discussed above).. In OCCT, component knowledge is incorporated through a constrained objective where specific voxel values must be equal to or greater than known attenuation coefficients, whereas KCR achieves such incorporation through the masking operation. The latter approach admits a straightforward extension to multiple known components without the need for explicit intersection detection. Both approaches utilize an alternating optimization scheme to solve for the respective objective functions, but employ different underlying algorithms. Specifically, for image updates: paraboloidal surrogates for KCR and so-called alternating minimization (AM) for OCCT; and for registration updates: quasi-Newton updates for KCR and various methods for OCCT. Deeper investigation is required to determine whether there are computational or convergence advantages for one approach over another.. One difference between KCR and OCCT involves the so-called "overshoots" in registration estimates discussed by Murphy et al. [27] . In that case, the algorithm was specifically modified using a dithering approach to avoid these overshoots and related streaking artifacts. Such issues were not observed with KCR. It may be that the penalty function in KCR effectively eliminates this problem; however, additional investigation is required to ascertain the underlying source.. One limitation of the basic KCR framework reported above is that the forward model employed in this initial implementation does not incorporate a polyenergetic beam and the physics (e.g., scatter) associated with a spectrum of X-rays. The mismatch between this monoenergetic algorithm and polyenergetic data will have an effect on convergence and residual artifacts. While these investigations are beyond the scope of this paper, an investigation of this mismatch and an incorporation of a polyenergetic forward model [41] [42] [43] [44] is the subject of ongoing work, with initial results found in [45] . Such modifications will likely be important to application in real clinical data since beam hardening is a major contributor to metal artifacts. Similarly, it is unclear how important the role of scatter will be, but it is straightforward to include a known scatter fraction in the forward model.. One potential limitation of this particular implementation of the KCR framework arises in cases where one has inexact knowledge of certain components-for example: metal rods used in vertebral fixation that are bent during the surgical procedure; knee or hip implants that have experienced significant wear prior to a surgical revision; or needle guidance in which the needle flexes upon insertion. However, we note that the mapping in (9) allows straightforward modification to a deformable registration model to encompass some of these scenarios (particularly when the differences can be expressed in a relatively low-dimensional parameter space). Such modifications to the model are underway and preliminary investigations have been reported in [46] .. The results summarized above focused largely on the image quality resulting from KCR; however, there are two outputs from the estimator: the image volume and the registration parameters. The image volume is likely the primary outcome, but the accurate pose estimation may also have clinical value in assessing the placement of components introduced in the body. For example, the pose output of the KCR method allows comparison of preoperative planning to the actual device delivery and may offer value in quantifying surgical precision.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Survivability in hierarchical telecommunications networks The survivable hierarchical telecommunications network design problem consists of locating concentrators, assigning user nodes to concentrators, and linking concentrators in a reliable backbone network. In this article, we study this problem when the backbone is 2-edge connected and when user nodes are linked to concentrators by a point-to-point access network. We formulate this problem as an integer linear program and present a facial study of the associated polytope. We describe valid inequalities and give sufficient conditions for these inequalities to be facet defining. We investigate the computational complexity of the corresponding separation problems. We propose some reduction operations to speed up the separation procedures. Finally, we devise a branch-and-cut algorithm based on these results and present the outcome of a computational study. Network design problems arising in telecommunications applications have originated new challenges in the field of optimization. Within the scope of this study is a two-layer telecommunication network infrastructure. In such a network, the traffic originating from the terminals (user nodes) is communicated through access networks to concentrators (switches or multiplexers) interconnected by a backbone (core) network. The traffic then traverses the backbone network and finally reaches the access network of its destination terminal through multiplexing at the backbone (the interested reader is referred to the surveys of [12] and [20] and the references therein). In the terminology of the hub location literature, the backbone network, carrying large volume of traffic at high speeds, takes the role of the hub network which enables the communication of the terminals or the demand centers. Because the backbone network is the primary means of providing communication between end-users, a reliable topological design is essential. Klincewicz [20] provides a classification of the underlying network design problems based on the topology of the access and the backbone networks. Typical access network topologies include stars, trees, and rings. Similarly, the backbone network may be a star, tree, complete, mesh, or ring.. This study focuses on designs where the access networks are stars and the backbone network is 2-edge connected. A graph is called k-edge connected for a non-negative integer k if it contains between any pair of distinct nodes at least k paths that do not share any edge (edge-disjoint). Figure 1 and star access networks; here, the squares represent the concentrators and the circles represent the terminals.. This article introduces an in depth analysis of the 2edge connected star subgraph problem (2ECSSP for short) to the telecommunications network design literature. In particular, we seek the most cost effective way of designing a backbone survivable telecommunications network by partitioning a given set of nodes into terminals and concentrators and establishing edges linking concentrators such that each terminal gets assigned to a single concentrator and the edges connecting the concentrators forming the backbone network becomes 2-edge connected.. Our problem inherently has two subproblems, namely the survivable network design problem and the concentrator location problem. Individually, both problems have been widely studied in the literature. Klincewicz [20] presents a survey for such problems. In his classification terminology of backbone network structure/access network structure, 2ECSSP becomes a 2-edge connected/star network design problem. Gourdin et al. [12] review the studies within the telecommunications context which include many variations of the concentrator location problems. Labbé et al. [23] consider the fully connected/star network design problem. Pirkul and Nagarajan [28] and Lee et al. [24] analyze the tree/star network design problem, whereas Gavish [11] studies the star/tree variant. Chardaire et al. [4] analyze the star/star network design problem. A path/path network design problem studied by Current and Pirkul [5] is another example of the two level network design approaches existing in the literature. However, except for the fully connected/star topology design, none of these designs guarantee survivability.. Survivable network design problems have been extensively studied; see for example [14, [16] [17] [18] 30] for surveys. In particular, the 2-edge connected subgraph problem, which is of close interest to 2ECSSP, is thoroughly investigated in the literature. Studies including [14, 17, 18, 25, 30, 31] but not restricted to this list consider the design of 2-edge connected survivable networks. Mahjoub [25] studies the polytope associated with the 2-edge connected subgraph problem, investigates its facets and shows that the polytope is completely described by the trivial and cut inequalities when the underlying graph is series-parallel. Barahona and Mahjoub [2] consider the 2-edge and 2-node connected subgraph problems on Halin graphs and provide the complete descriptions of the associated polytopes. If the 2-edge connected subgraph polytope is completely described by the cut and trivial inequalities for a graph, then the graph is called perfectly 2-edge connected and the problem can be solved in polynomial time. Mahjoub [26] introduces new classes for such graphs and provides sufficient conditions for a graph to be perfectly 2-edge connected. Fonlupt and Mahjoub [7] introduce and study the concept of critical extreme points for the 2-edge connected subgraph polytope. In particular, they characterize the perfectly 2-edge connected graphs.. Several extensions of the 2-edge connected subgraph problem have been investigated in the literature. Kerivin et al. [19] consider the (1,2)-survivable network design problem. Here each node is assigned a connectivity type 1 or 2, and the problem is to determine a subgraph such that between every pair of nodes (s, t), there are at least min{r(s), r(t)} edge-disjoint paths. This is a special case of a more general model introduced by Grötschel et al. [14] . Vandenbussche and Nemhauser [31] study the 2-edge connected subgraph problem on graphs when an edge may be used more than once. Fortz et al. [8, 9] study the problem of designing 2-nodeconnected networks with bounded meshes. The network to be designed is 2-node connected and every edge of the network must be contained in a cycle whose length is bounded from above. They describe several classes of valid inequalities and propose a branch-and-cut algorithm. Fortz et al. [10] study a similar problem in which the network to be designed is 2-edge connected. Further work on the k-edge connected subgraph problem with (and without) hop constraints can be found in [3, 6, 16] .. Labbé et al. [21] study the ring/star network design problem which is of close kinship to the 2ECSSP. Indeed, this problem is a restriction of the 2ECSSP as a ring is a 2-edge connected network. The authors identify some facet defining inequalities of the associated polytope. A branch-and-cut algorithm is developed based on these inequalities and computational results are presented. We note here that as 2ECSSP is a relaxation of the ring/star network design problem, any valid inequality for the 2ECSSP polytope is also valid for the ring/star polytope; however, the converse is not necessarily true.. This work contributes to the two level network design literature by expanding it with the 2ECSSP, a special survivable infrastructure that has not yet been analyzed in the literature. The contribution includes a 0-1 model development and a detailed polyhedral analysis of the associated polytope. For the families of facet defining inequalities, the computational complexity status of each separation problem is established and exact and/or heuristic separation algorithms are designed. To speed up the separation runtime, some reduction operations are proposed and analyzed. Finally, a branch-and-cut algorithm which assembles all this theoretical development is designed for the 2ECSSP. This article is organized as follows. In the following section, we give an integer programming formulation for the 2ECSSP. In Section 3, the polytope associated with 2ECSSP is analyzed and several classes of valid inequalities for the 2ECSSP are derived. We give necessary and sufficient conditions for these inequalities to be facet defining. The reduction operations are proposed in Section 4, and the separation algorithms are discussed in Section 5. In Section 6, we present the computational study and finally we give some concluding remarks in Section 7. In this article, we studied 2ECSSP, which is a special case of a survivable hierarchical telecommunications network design problem. We presented some valid inequalities for the associated polytope and described the conditions for them to be facet defining. The separation problems for the valid inequalities and some reduction operations are discussed. We proposed some heuristic algorithms for separation problems. Based on these findings, a branch-and-cut algorithm is developed.. Our computational analysis showed that the extended F-partition and star-path inequalities are important for the success of the branch-and-cut algorithm. It is also observed that the heuristic algorithms used for separation problems and the reduction operations are effective in the branch-andcut algorithm. Our experiments showed that the number of branch-and-cut nodes is higher when α is smaller, but it also takes less time to find the optimal solution in this case.. In this article, we only studied survivability of the backbone network in a two level telecommunications network. It may be interesting to incorporate survivability for local access networks. Another interesting extension may be to include the installation of capacities for routing the demands in the network in addition to the survivability aspect.

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O