On the Amortized Complexity of Zero-Knowledge Protocols Abstract. We propose a general technique that allows improving the complexity of zero-knowledge protocols for a large class of problems where previously the best known solution was a simple cut-and-choose style protocol, i.e., where the size of a proof for problem instance x and error probability 2 −n was O(|x|n) bits. By using our technique to prove n instances simultaneously, we can bring down the proof size per instance to O(|x| + n) bits for the same error probability while using no computational assumptions. Examples where our technique applies include proofs for quadratic residuosity, proofs of subgroup membership and knowledge of discrete logarithms in groups of unknown order, interval proofs of the latter, and proofs of plaintext knowledge for various types of homomorphic encryption schemes. We first propose our protocols as Σ-protocols and extend them later to zero-knowledge proofs of knowledge. In a zero-knowledge protocol, a prover tries to convince a skeptical verifier that a certain statement is true. Except with a small error probability, the verifier should be convinced if and only the statement is indeed true, but should learn nothing beyond the validity of the assertion. The statement can take the form of claiming that the input string x is in a given language L (interactive proofs) or claiming that the prover knows a "witness" w such that (x, w) is in some given relation R (interactive proofs of knowledge).. Zero-knowledge was introduced in [8] and has been the subject of intense research ever since. Zero-knowledge protocols are interesting as theoretical objects in their own right but are also very useful as building blocks in larger protocols.. The efficiency of zero-knowledge proofs has been studied in many works, and in some cases, extremely efficient zero-knowledge proofs have been found. For NP complete problems such as Boolean circuit satisfiability, Ishai et al. [10] show protocols where the proof size (communication complexity) is O(|x| + poly(k)), where k is a security parameter, and |x| is the size of the input x. In prime-order groups, Schnorr's protocol [11] proves knowledge of a discrete logarithm using a (honest verifier) zeroknowledge proof of size O(|x| + n) for an error probability of 2 −n . In a practical application, one must expect to have to communicate x to make the claim in the first place as well as an n-bit challenge, so this is essentially optimal. 1 However, for several interesting problems, such methods for improved zero-knowledge protocols do not work. This includes, for instance, the very first problem for which a zero-knowledge protocol was suggested, namely quadratic residuosity, where one proves on input (x, N ) that x is a square modulo N (and that the prover knows a square root). The well-known classical protocol [4] for this has error probability 1/2, and one must repeat it n times for an error probability of 2 −n so that the proof will be of size O (|x|n) . No more efficient solution with unconditional soundness and zero-knowledge was previously known.. The state of affairs is similar for the discrete log problem in groups of unknown order. Say we are given g, h ∈ Z * N for an RSA modulus N , and the prover claims that h is in the group generated by g and that he knows the discrete logarithm of h base g. The best solution we know for this has error probability 1/2, and again we must repeat the entire protocol to reduce the error. Schnorr's protocol cannot be used here since its proof of soundness requires that the group order is known, and finding the order of Z * N is equivalent to factoring N . Even if we were happy with only a proof of membership in the group generated by g, the error probability for known solutions would be 1 divided by the smallest prime factor in the total group order, which is 1/2 for the case of Z * N . It should be noted that Fujisaki and Okamoto [7] have shown how to get around these difficulties, but only if we are guaranteed to be working in a subgroup of Z * N with only large prime factors in the order, and then only under the strong RSA assumption.. Other examples of a similar nature come from various proposals for homomorphic encryption where one uses subgroups of Z * N , often with small prime factors in their order, to make the decryption algorithm efficient [5, 9] . Many applications require a zero-knowledge proof that one knows the plaintext for a given ciphertext, and we then have all the same problems as described above.. In this paper, we show a general method that applies to all the problems mentioned above and allows us to reduce the proof size in the amortized sense: we can give a proof for n instances of the problem simultaneously such that the communication complexity per instance proved is O(|x| + n) for an error probability of 2 −n , thus we are as efficient as the best known unconditional protocols for any problem. The technique uses no computational assumptions. In all cases, the computational complexity is also reduced compared to naive repetition of the basic protocol. Here, the most favorable case is discrete log where computation is reduced by a factor n, for quadratic residuosity we gain a factor log n.. We emphasize that for the case of proofs for quadratic residues, what we achieve is different from what can be done using Fiat-Shamir-type protocols [6] , although they may seem superficially similar to ours: the Fiat-Shamir protocol is also efficient, it takes n quadratic residues as input, and has an error probability of 2 −n . The difference lies in the type of witness that the prover proves knowledge of. For Fiat-Shamir, the prover only has to know a product of some of the square roots of the input numbers. In fact, the prover could know nothing about the first input number and still survive with probability 1/2. In contrast, our protocol guarantees that the prover knows all the square roots.. It also makes sense to compare our work to the "MPC-in-the-head" technique from [10] , where the idea is to construct a zero-knowledge protocol from a multiparty computation protocol. The technique requires a commitment scheme, and it is well known that such schemes cannot offer unconditional security for both parties. This means that for any protocol constructed using MPC-in-the-head, either soundness or zero-knowledge will be computational, unlike our results that require no computational assumptions. Nevertheless, it should be noted that one can adapt the construction from [10] to give a protocol for quadratic residuosity modulo some number N with complexity similar to ours. This is because one can start the construction of [10] from a multiparty protocol that is particularly efficient because it is based on a special-purpose secret sharing scheme that works modulo N . For other cases, such as discrete log in a group of unknown order, no such secret-sharing scheme is available, one has to resort to generic MPC protocols, and the complexity then becomes much larger than ours.. We give an abstract framework characterizing the type of problem that our method applies to and derive all the examples above as special cases. Basically, we need a function with certain homomorphic properties on some Abelian groups and a ring A that acts on the groups in a sufficiently nice way. The latter holds if all groups are Amodules (although this is not a necessary condition). It follows that Z can be always used as A since all Abelian groups are Z-modules.. Applications of our result include multiparty computation based on homomorphic encryption, where players would supply inputs by sending them in encrypted form. To make the overall protocol secure, players must prove that they know the inputs they supply, and our method can be used to give such proofs efficiently for a large number of ciphertexts, see [3] for a concrete example of such an application. Note that some computations, such as certain auctions, do in fact require players to submit large amounts of data as input. Another application involves proofs of negative statements such as proving that a number x is not a square modulo N . The classical protocol for this from [8] uses the proof for quadratic residuosity as a subroutine and has complexity O(|x|n 2 ). Our method reduces this to O(|x|n) without making any computational assumptions.. The same idea can used to prove for some homomorphic encryption schemes that a ciphertext contains a nonzero plaintext. Note that these applications are for a single instance proof and so are not of an amortized nature.. The Σ-protocols presented in Sect. 3.4 are only honest-verifier zero-knowledge (HVZK), as it is the case for Schnorr's protocol. In Sect. 5, we present an extended version of our basic protocol (called Protocol 1 in the following), which is a zeroknowledge proof of knowledge. We use a technique by Bellare et al. [2] to construct it but extend their result by showing soundness as a proof of knowledge, which was not done in [2] . As common for zero-knowledge proofs, the verifier has to commit to his challenge at the beginning of the protocol. Here, the commitment scheme is constructed from basic Σ -protocol for a new instance generated by the prover. The extended protocol has only constant-factor overhead of the basic one.

O O O O O B B O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Towards identity anonymization on graphs The proliferation of network data in various application domains has raised privacy concerns for the individuals involved. Recent studies show that simply removing the identities of the nodes before publishing the graph/social network data does not guarantee privacy. The structure of the graph itself, and in its basic form the degree of the nodes, can be revealing the identities of individuals. To address this issue, we study a specific graph-anonymization problem. We call a graph k-degree anonymous if for every node v, there exist at least k −1 other nodes in the graph with the same degree as v. This definition of anonymity prevents the re-identification of individuals by adversaries with a priori knowledge of the degree of certain nodes. We formally define the graph-anonymization problem that, given a graph G, asks for the k-degree anonymous graph that stems from G with the minimum number of graph-modification operations. We devise simple and efficient algorithms for solving this problem. Our algorithms are based on principles related to the realizability of degree sequences. We apply our methods to a large spectrum of synthetic and real datasets and demonstrate their efficiency and practical utility. Social networks, online communities, peer-to-peer file sharing and telecommunication systems can be modelled as complex graphs. These graphs are of significant importance in various application domains such as marketing, psychology, epidemiology and homeland security. The management and analysis of these graphs is a recurring theme with increasPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ing interest in the database, data mining and theory communities. Past and ongoing research in this direction has revealed interesting properties of the data and presented efficient ways of maintaining, querying and updating them. However, with the exception of some recent work [2, 10, 19, 14, 18] , the privacy concerns associated with graph-data analysis and management have been largely ignored.. In their recent work, Backstrom et. al. [2] point out that the simple technique of anonymizing graphs by removing the identities of the nodes before publishing the actual graph does not always guarantee privacy. It is shown in [2] that there exist adversaries that can infer the identity of the nodes by solving a set of restricted graph isomorphism problems. However, the problem of designing techniques that could protect individuals' privacy has not been addressed in [2] .. Hay et. al. [10] further observe that the structural similarity of nodes' neighborhood in the graph determines the extent to which an individual in the network can be distinguished. This structural information is closely related to the degrees of the nodes and their neighbors. Along this direction, the authors propose an anonymity model for social networks -a graph satisfies k-candidate anonymity if for every structure query over the graph, there exist at least k nodes that match the query. The structure queries check the existence of neighbors of a node or the structure of the subgraph in the vicinity of a node. However, the authors of [10] mostly focus on providing a set of anonymity definitions and studying their properties, and not on designing algorithms that guarantee the construction of a graph that satisfies their anonymity requirements.. Motivated by Backstrom et al. and Hay et al.' s work, we want to answer the following question that is still unanswered: how to minimally modify the graph to protect the identity of each individual involved? Being able to answer this kind of questions is important to understand the privacy issues on graph/social networks, the computational complexity of the related problems, and other challenges and opportunities in this emerging field (see e.g., Jon Kleinberg, Challenges in Social Network Data: Processes, Privacy and Paradoxes, ACM SIGKDD 2007 Invited Talk) . We note that in a social network, nodes correspond to individuals or other social entities, and edges correspond to social relationships between them. The privacy breaches in social network data can be grouped to three categories: 1) identity disclosure: the identity of the individual who is associated with the node is revealed; 2) link disclosure: sensitive relationships between two individuals are disclosed; and 3) content disclosure: the privacy of the data associated with each node is breached, e.g., the email message sent and/or received by the individuals in a email communication graph. We believe that a perfect privacy-protection system should consider all of these issues. However, protecting against each of the above breaches may require different techniques. For example, for content disclosure, standard privacy-preserving data mining techniques [1] , such as data perturbation and k-anonymization can help. For link disclosure, the various techniques studied by the link-mining community [8, 19] can be useful.. In this paper, we focus on identity disclosure. We propose a systematic framework for identity anonymization on graphs. We use some of the definitions proposed in [10] , and we solve the algorithmic problems that arise from these definitions. The degree of a node in a graph, among other structural characteristics, can to a large extent distinguish the node from other nodes. In this paper, we studied a specific graphanonymity notion that prevents the re-identification of individuals by an attacker with certain prior knowledge of the degrees. We formally defined the Graph Anonymization problem that, given an input graph asks for the minimum number of edge additions (or deletions) that allow the transformation of the input to a degree-anonymous graph; i.e., a graph in which every node shares the same degree with k − 1 other nodes. We decomposed this problem into two subproblems and proposed simple and efficient algorithms for solving them. We applied our algorithms to a set of synthetic and real-world graph data and demonstrated the utility of the degree-anonymous graphs as well as the efficiency of our methods. Finally, we extended our algorithms to allow simultaneous edge additions and deletions.. Before concluding this paper, we would like to note that, compared with existing data anonymization and perturbation techniques for tabular data, dealing with graphs is a much more challenging task. In tabular data, each tuple can be viewed as an independent sample from some distribution. However, in a graph, all the nodes and edges are correlated; a single change of an edge and/or a node can spread across the whole network. Moreover, in graphs it is difficult to model the capability of an attacker. Any topological structure of the graph can be potentially used to derive private information. Finally, it is difficult to measure the utility of a graph. We are not aware of any effective metrics to quantify the information loss incurred by the changes of its nodes and edges.. In this paper, we tried to address some of these issues using simple and intuitive notions. Lots of additional work needs to be done in order to develop theoretically and practically sound privacy models for graphs. Since the introduction of the concept of anonymity in databases [15] , there has been increasing interest in the database community in studying the complexity of the problem and proposing algorithms for anonymizing data records under different anonymization models [4, 12, 13] . Though lots of attention has been given to the anonymization of tabular data, the privacy issues of graphs/social networks and the notion of anonymization of graphs have only been recently touched.. Backstrom et. al. [2] show that simply removing the identifiers of the nodes does not always guarantee privacy. Adversaries can infer the identity of the nodes by solving a set of restricted isomorphism problems based on the uniqueness of small random subgraphs embedded in an network.. Hay et. al. [10] observe that the structural similarity of the nodes in the graph determines the extent to which an individual in the network can be distinguished from others. Based on the notion of k-anonymity [15] , they propose a k-candidate anonymity model. Pei and Zhou in [14] consider yet another definition of graph anonymity -a graph is k-anonymous if for every node there exist at least k − 1 other nodes that share isomorphic neighborhoods; in this case the neighborhood of a node is defined by its immediate neighbors and the connections between them. This definition of anonymity in graphs is different from ours. In a sense it is a more strict one. Given the difference in the definition, the corresponding algorithmic problems arising in [14] are also different from the problems we consider in this paper.. Zheleva and Getoor [19] consider the problem of protecting sensitive relationships among the individuals in the anonymized social network. This is closely related to the linkprediction problem that has been widely studied in the linkmining community [8] . In [19] simple edge-deletion and node-merging algorithms are proposed to reduce the risk of sensitive link disclosure. This work is different from ours in that we are primarily interested in identity protection; once the identity of each individual is disguised, there is no need to hide the sensitive relationships any more. Also the set of combinatorial problems that we need to solve in our framework are very different from the set of problems discussed in [19] .. Sensitive link and relationship protection is also discussed by Ying and Wu [18] . They study how anonymization algorithms that are based on randomly adding and removing edges change certain graph properties. More specifically, they focus on the change caused in the eigenvalues (spectrum) of the network. The authors additionally explore, how the randomized network can be exploited by the adversary to gain knowledge about the existence of certain links. The focus of our paper as well as the set of techniques we develop are orthogonal to those discussed in [18] .. Frikken and Golle [7] study the problem of assembling pieces of graphs owned by different parties privately. They propose a set of cryptographic protocols that allow a group of authorities to jointly reconstruct a graph without revealing the identity of the nodes. The graph thus constructed is isomorphic to a perturbed version of the original graph. The perturbation consists of addition and or deletions of nodes and or edges. Unlike their work, we try to anonymize a single graph while keeping the data utility in mind. Moreover, the methods in [7] involve cryptographic protocols that are computationally demanding, while our approach employs simple and efficient combinatorial algorithms. In this section we evaluate the performance of the proposed graph-anonymization algorithms.

O O O O O O O O B B O O O O O O B O O O O O O O O O O O O O O O O O O O O B B O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O B O B O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O B O O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O B O B B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A DOMAIN-SPECIFIC LANGUAGE TO DESIGN ENTERPRISE APPLICATION INTEGRATION SOLUTIONS Enterprise Application Integration (EAI) solutions cope with two kinds of problems within software ecosystems, namely: keeping a number of application's data in synchrony or creating new functionality on top of them. ESBs provide the technology required to implement a variety of EAI solutions at sensible costs, but they are still far from negligible. It is not surprising then that many authors are working on proposals to endow them with domain-specific tools to help software engineers reduce integration costs. In this article, we introduce a proposal called Guaraná. Its key features are as follows: it provides explicit support to devise EAI solutions using enterprise integration patterns by means of a graphical model; its DSL enables software engineers to have not only the view of a process, but also a view of the whole set of processes of which an EAI solution is composed; both processes and tasks can have multiple inputs and multiple outputs; and, finally, its runtime system provides a task-based execution model that is usually more efficient than the process-based execution models in current use. We have also implemented a graphical editor for our DSL and a set of scripts to transform our models into Java code ready to be compiled and executed. To set up a solution from this code a software engineer only needs to configure a number of adapters to communicate with the applications being integrated. Nowadays, most companies run many applications in their software ecosystems 20 to carry out their business activities. These applications are frequently software packages purchased from third parties or legacy systems. ment the previous abstractions and cope with the problems and constraints that were mentioned previously. Typical ESBs provide so-called adapters or connectors, which are used to communicate with the applications being integrated, and an orchestration language to define workflows of messages amongst processes and/or applications. Java Business Integration (JBI) is an specification of a pluggable architecture of services 2 to which several ESBs have adhered, e.g., Open ESB, Petals ESB, ServiceMix or its commercial version, Fuse ESB 24 . In JBI, adapters and connectors are referred to as binding components, and the orchestration language used is the Business Process Execution Language (BPEL) 25 . There are many types of binding components available nowadays, which allows solutions to connect to almost any existing application. The catalogue includes binding components for databases, files, SOAP/HTTP, RSS, SMTP, RMI/IIOP, JMS, HL7, LDAP, DCOM, and so on. Our research focuses on Open ESB.. Although BPEL is quite a common orchestration language, it is too general. It provides only general constructs like invoke, receive, reply, assign, throw, wait, or compensate. It does not provide explicit support to the whole set of the well-known Enterprise Integration Patterns (EIPs) 15 that are used extensively in the EAI Community. A few EIPs can be implemented directly as part of an orchestration process, e.g., filters, routers or mergers; many others, however, need to be implemented separately as independent services that need to be invoked by an orchestration 33 . This results obviously in inefficiency, but also in difficulties to understand and maintain the resulting solutions. This has motivated many software engineers to use Camel 16 or Spring Integration 8 , since these tools help use many EIPs inside orchestration processes; obviously, they do not rely on BPEL.. Model Driven Development (MDD) is an approach to software development in which models are first-class citizens and the centre of the development process 13 . Domain-Specific Languages (DSLs) play an important role in MDD. Every DSL is supported by an abstract syntax, aka, metamodel, that provides a number of concepts that can be used to devise solutions to problems in a given domain; the solutions are specified by means of a concrete syntax that allows to represent the concepts in the metamodel textually or graphically; such solutions are usually referred to as models, and the key is that they must be accompanied by a number of transformations to translate them into code written in a programming language. Overall, using models allows software engineers to devise solutions to their problems at a proper abstraction level and relieves them from the burden of dealing with the rather low-level constructs provided by programming languages.. In this article, we present a proposal called Guaraná. Its main advantages with regard to similar proposals are as follows: it provides explicit support to devise EAI solutions using enterprise integration patterns by means of a graphical model; its DSL is graphical, and it enables software engineers devise EAI solutions at a high-level of abstraction; not only provides the DSL the view of a process, but also a view of the whole set of processes of which EAI solutions are composed; both processes and tasks can have multiple inputs and multiple outputs; and, finally, its runtime system provides a task-based execution model that is usually more efficient than the process-based execution models in current use. We have also implemented a graphical editor for our DSL a set of scripts to transform our models into Java code ready to be compiled and executed. To set up a solution from this code a software engineer only needs to configure a number of adapters to communicate with the applications being integrated. Our results were implemented in the laboratory using the following tools: Eclipse Helios as the workbench, the Eclipse Modelling Framework (EMF) 29 to implement the metamodel, the OCLInECore plug-in 32 to implement constraints in the metamodel, the EuGENia plug-in 18 and the Graphical Modeling Framework (GMF) 9 to implement the editor, the Dresden OCL Toolkit 6 to validate the OCL specification, and MOFScript 22 to implement our model-totext transformations. This implementation helped us validate our proposal on a non-trivial, real-world case study.. The rest of this article is structured as follows: Section §2 presents the related work and compares other proposals to ours; Section §3 presents Guaraná in a nutshell so that readers can have an overall idea of the proposal; Section §4, presents the abstract syntax of our DSL, and Section §5 complements it with a concrete syntax; Section §6 reports on the runtime system that supports our proposal; Section §7 summarises the transformations we have devised to translate Guaraná into Java code; Section §8 summarises one of the task toolboxes we have devised for Guaraná; Section §9 reports on a validation we have performed using a non-trivial, real-world integration problem; finally, Section §10 reports on our main conclusions. Companies are increasingly relying on ESBs to implement EAI solutions. They provide a number of binding components software engineers can use to connect to almost every existing application and a language to design orchestration processes. The most common such language is BPEL; unfortunately, it is rather low-level, since it does not support directly many common enterprise integration patterns. This has motivated many authors to work on domain-specific tools to endow ESBs with the appropriate abstraction level to model EAI solutions. There are many proposals in the literature, of which Camel, Spring Integration and BizTalk are the most closely related to ours.. In this article, we have introduced Guaraná, which is a proposal that aims at increasing the level of abstraction of ESBs, in an attempt to make it easier for software engineers to devise, implement, and deploy their EAI solutions. Guaraná is classified as an external DSL. We have presented Guaraná's abstract and concrete syntaxes, we have described the runtime system that supports the proposal, the transformations that translate Guaraná models into Java code ready to be compiled and executed, and a generic task toolbox; we have also reported on the results of a project we have carried out to validate our proposal. We have also compared our proposal to others in the literature, and our conclusion is that Guaraná's key features are the following: it provides explicit support to devise EAI solutions using enterprise integration patterns by means of a graphical model; its DSL is graphical, and it enables software engineers devise EAI solutions at a high-level of abstraction; not only provides the DSL the view of a process, but also a view of the whole set of processes of which EAI solutions are composed; both processes and tasks can have multiple inputs and multiple outputs; and, finally, its runtime system provides a task-based execution model that is usually more efficient than the process-based execution models in current use. We have also implemented a graphical editor for our DSL a set of scripts to transform our models into Java code ready to be compiled and executed. The catalogue of Enterprise Integration Patterns (EIPs) proposed by Hohpe and Woolf 15 puts a foundation to the majority of proposals in the field of EAI, including ours. This catalogue describes a number of recurring problems in EAI and a number of proposals to solve them. The catalogue is informal, in the sense that it is not intended to provide a tool by means of which a software engineer can devise, build and deploy an EAI solution; contrarily, it can be viewed as a cookbook that a software engineer can use to have a better understanding of common solutions to common EAI problems, without committing to a particular technology or tool.. Scheibler et al. 27 presented an approach to enable the use of EIPs for EAI in the context of the Software as a Service (SaaS) business model. They claimed that this model relieves software engineers from setting up the environment where an EAI solution is executed, and the same pattern implementation can be reused by different solutions once they are provided as services. Although it is an interesting approach for EAI, it differs from ours because it is narrowly focused on the SaaS business model, which is not our focus.. Dirgahayu et al. 7 introduced a language and a set of patterns to design EAI solutions in the context of web services. Their focus was on the design of orchestration processes and how to represent their interactions. They do not provide explicit support for EIPs, but model EAI solutions as compositions of processes that interact amongst themselves and/or with the applications being integrated. The authors mentioned that this is important to increase the level of abstraction and to enable business analysts to participate actively in the design of EAI solutions. Yuan 33 presented a tool that helps generate code for EIPs automatically. EIPs are configured with their inputs and outputs and the tool maps them onto BPEL processes. The author stresses that it is not usually possible to implement EIPs relying solely on BPEL constructs; in such cases, a part of the EIP must be implemented as independent web services. Unfortunately, this proposal does not provide a DSL like ours, but only a tool to translate some EIPs.. Camel 16 and Spring Integration 8 provide fluent APIs 10 by means of which it is possible to implement solutions using constructors that are very close to the EIPs; however, they are totally bound to the Java technology. Therefore, they do not provide the same level of abstraction as our DSL. From the analysis of the literature and tools, we conclude that Camel, Spring Integration, and BizTalk are the most closely related proposals. To compare Guaraná to these tools, we have built a comparison framework with a set of objective properties that are classified into scope, modelling, and technical properties. We report on the comparison framework in the following subsections.

O B I O O B I I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


From data to knowledge mining Most past approaches to data mining have been based on association rules. However, the simple application of association rules usually only changes the user's problem from dealing with millions of data points to dealing with thousands of rules. Although this may somewhat reduce the scale of the problem, it is not a completely satisfactory solution. This paper presents a new data mining technique, called knowledge cohesion (KC), which takes into account a domain ontology and the user's interest in exploring certain data sets to extract knowledge, in the form of semantic nets, from large data sets. The KC method has been successfully applied to mine causal relations from oil platform accident reports. In a comparison with association rule techniques for the same domain, KC has shown a significant improvement in the extraction of relevant knowledge, using processing complexity and knowledge manageability as the evaluation criteria. We live in a world of fast generation and distribution of enormous amounts of information. One way of dealing with that is simply to randomly pick a manageable amount of information and react to it. However, this strategy is not well suited to the business world, given that knowledge is crucial to survival in a competitive environment. This creates a need for automated information extraction techniques.. Any engineering field creates artifacts such as products or processes. During an artifact's life cycle a great deal of data is produced, starting with an initial set of specifications and continuing through maintenance reports. Many of these documents are textual. Data management technology has evolved and has provided the means for organizing and storing information. Having information available is a necessary, but not sufficient, condition for learning. People must be able to retrieve and digest information to craft new knowledge from the stored information. Unfortunately, making sense of engineering project data, even for a specific domain area or company, overwhelms human capabilities. Decision-making processes improve whenever successful and unsuccessful cases are understood, revealing flawed decision patterns that should be avoided (Maher & Garza, 1995; Soibelman & Kim, 2000) .. Despite the difficulties, the importance of learning from the past has led researchers to carefully look for analytic techniques to handle large amounts of engineering data. In this scenario, the use of data mining (DM) techniques has become an attractive alternative to classic mathematical models when applied to specific tasks such as problem diagnosis and failure prediction (Varde et al., 2007) . DM is the process of discovering relevant knowledge from large data repositories (Fayyad et al., 1996) . Although this assertion is broadly accepted, making this process work is still a challenge. There are many DM algorithms that differ in the type of information they extract, ranging from association rules to classifications or patterns. One of the most popular DM strategies is the association rules technique (Agrawal & Srikant, 1994) , which takes a data set of items, and generates IF-THEN rules based on relations of co-occurrence between elements. Unfortunately, association rule techniques still fail to produce a manageable amount of relevant information, usually producing too many results, which include much redundancy.. A large amount of research has been done on DM, most of it applied to classification tasks, such as identifying patterns of good and bad maintenance strategies, knowledge transfer from maintenance to design (Jadhav et al., 2007) , identifying calibration models in building structural components (Saitta et al., 2005) , and eliciting the customer's project requirements (Lo et al., 2007) . Refrat et al. (2004) investigated the benefits of using DM techniques to assist facility managers in the identification of potential building maintenance problems given data from building air handling units and thermostatic mixing valves. Maintenance data were organized in a database. Five thousand work orders for a specific building, collected during a period of approximately 2 years, were used. Classic statistical methods and different DM techniques such as association rules, clustering algorithms, and decision trees were applied. They concluded that DM techniques support the identification of critical cost issues, as well as the creation of predictive plans. Jadhav et al. (2007) used classification DM techniques to identify the causes of high severity events that lead to major financial losses in aeroengine artifacts. They focused on providing answers to a predefined set of maintenance questions that could improve posterior design. Information was organized in a data warehouse, which makes the mining process easier because such data contains no noise. Saitta et al. (2005) investigated an innovative use of DM for system identification, also known as model calibration, in structural engineering. Classic approaches (Modak et al., 2002; Koh et al., 2003) have been based on calculations of structural stiffness coefficients to predict structural behavior. They took a different approach, using DM techniques to reveal properties of good models that could explain observations of structural behavior and create models from these observations. Chen et al. (2003) claimed that product design would benefit from prior customer feedback and marketing analysis. They investigated the use of neural networks to elicit user requirements for the design of wooden golf clubs. Their approach requires an input matrix of previously recorded graded imposed constructs that reflect the perception of the product, with an emphasis on requirements originating from multicultural differences among users.

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Keyphrase extraction through query performance prediction Previous research shows that keyphrases are useful tools in document retrieval and navigation. While these point to a relation between keyphrases and document retrieval performance, no other work uses this relationship to identify keyphrases of a given document. This work aims to establish a link between the problems of Query Performance Prediction (QPP) and keyphrase extraction. To this end, features used in QPP are evaluated in Keyphrase Extraction using a Naïve Bayes classifier. Our experiments indicate that these features improve keyphrase extraction effectiveness in documents of different length. More importantly, commonly used features of frequency and first position in text perform poorly on shorter documents, whereas QPP features are more robust and achieve better results. Keyphrases are phrases formed of one or more terms that reflect the salient content of a document. The term keyphrase is used instead of more common expression "keyword" to emphasize that keyphrases can be formed of multiple terms. A user attempting to fulfil an information need issues a query to a full-text search engine and scans each retrieved document to have at least a basic understanding of its contents. In this vein, keyphrases are helpful in terms of shortening the time spent, as they are concise representations of the content. Nonetheless, most of the electronically available documents do not have keyphrases assigned to them. One approach to this problem is building a keyphrase extraction system that automatically determines the keyphrases of a given document.. A typical keyphrase extraction system forms a list of candidates from the phrases that appear in the original document, and evaluates each candidate using features acquired from the text. Two commonly used features are frequency and first occurrence position of the phrase, which are referred to as in -document features in this article. This research through experiments shows that in-document features are not as effective in short documents as they are in long documents. Following from this, it proposes a method using novel features to achieve more robust results for documents of different lengths.. Our method is motivated by the observation that when a keyphrase is searched in a corpus, the retrieved set of documents is relatively focused on a single domain with high similarity. For example, while "machine" and "learning" are two ambiguous terms that appear in documents from different domains, the phrase "machine learning" appears in a document set that is more concentrated on a single domain. In this context, keyphrases are expected to be good queries, and are utilized in document retrieval and browsing. Phrasier [1] creates an index using only the keyphrases of documents instead of the full-text, and reports no negative impact on document retrieval performance. Furthermore, Most of the earlier works on keyphrase extraction focus es on research articles. However, there is an increasing interest on applying keyphrase extraction in shorter documents such as Twitter messages [35] and news articles [12] . In this research, potential problems of features commonly used in keyphrase extraction were shown through experiments. Although these features are useful in full-text articles, their effectiveness drops in short documents. Features extracted from a background corpus are able to solve this problem. We have shown that while the introduced QPP features improve keyphrase extraction in full-text articles, the improvement is much more considerable for shorter documents like abstracts. Furthermore, our features are not dependent on the occurrences of the phrases in the original document, and can be calculated for phrases that never appear in the document. All in all, this work aimed to establish a link between the problems of QPP and keyphrase extraction. We believe that this work contributes to the research on finding keyphrases by removing the constraints imposed by features directly extracted from the occurrences in the original document. A careful investigation of techniques for creating candidate keyphrase lists by mining related articles or semantically related phrases enables our algorithm to generate keyphrases. The tec hniques used in this article may lead to more general methods that are able to operate on different genres and perform generation instead of extraction. Keyphrase extraction systems are usually evaluated using precision and recall, which are defined in terms of sets of phrases. In a processed source document, let the set A be the author assigned phrases, A' be the subset of A formed of phrases appearing in the document. Let E be the set of phrases extracted automatically by the system. Equation 14 shows the calculation of recall. To avoid penalizing keyphrase extraction systems for keyphrases that cannot be extracted, the recall value is calculated with res pect to the set A'. Precision is calculated as shown in Equation 15 .. The test and training data are chosen in a way to be compatible with the experiments performed in Turney [6] and Frank et al. [7] . 20 of the journal articles are reserved for testing and the remaining 55 documents are used for training. In the corpus of abstracts, 53 documents are used for training and 19 for testing, where 3 have been omitted as they lack an abstract. Tables 3 and 4 present the results of the full-text and abstract experiments respectively. For both of the experiments, the precision, recall and average number of correct keyphrases per-document are given when 5, 10 and 15 keyphrases are extracted for each article. The results of Kea [7] 5 are also provided for comparison. In Tables 3 and 4 , inDoc+QPP denotes the experiments using all of the features defined in Table 1 . The feature set inDoc denotes tf*idf and firstPosition. QPPFeats denotes all features, excluding those of inDoc. In full-text articles, Kea algorithm is able to extract 38% of the keyphrases appearing in the articles, when 15 keyphrases are extracted for each document. Journal articles concisely define the contribution of the document in early sections, and keyphrases are used more frequently in abstracts and introduction portions of the document. This is why the firstPosition feature achieves high accuracy in scientific articles.. As seen in Table 3 , the effectiveness of QPPFeats is lower than Kea and inDoc. We have observed that QPP features tend to have similar values for domain specific phrases, keyphrases and sub or super phrases of keyphrases. In fact, for a superset of a keyphrase a similar set of documents is usually retrieved from the background corpus. For example, for the keyphrase "obsessive compulsive disorder" and the sub -phrases "obsessive compulsive", "compulsive disorder" as well as the super-phrase "obsessive compulsive personality disorder", similar sets of documents are retrieved from the background corpus. Since all QPP features are calculated using the retrieved documents, the feature values are almost identical to each other. In order to tackle this problem and to improve the effectiveness of the system, we integrated QPP features with in-document features in full text articles. As a result, inDoc+QPPFeats system achieves the best recall values when compared to Kea and inDoc algorithms.. The results of the experiment using abstracts, as shown in Table 4 , reveal a defect of inDoc features in shorter documents. While tf*idf and firstPosition are able to achieve high precision and recall values in full-text experiments, their performance is poor in the abstract corpus. As indicated previously, the firstPosition feature, depending on the structure of the document, is effective in full-text articles. However, in shorter documents the firstPosition, which is the normalized distance from the start of the document to the word, is subject to more noise. While a change in distance by few words does not change the value of firstPosition in long documents, it changes the distance value significantly in short documents. tf*idf values are formed of two components of term frequency and inverse document frequency. Inverse document frequency gets larger values when the phrase occurs in fewer documents. In short texts, most phrases occur once or a couple of times. Since frequencies of terms are similar, a high value of tf*idf is assigned to a phrase that appears infrequently in the corpora. Thus, for short documents it is even possible to observe the highest tf*idf values in spelling errors and typos.. The QPP features are not extracted directly from the document, and can be calculated for any phrase regardless of how many times it occurs in the text, if it ever does. This makes them more robust to the changes in the length of the documents. In the case of abstracts, the recall values of QPPFeats+inDoc and QPPFeats are better than those of the Kea algorithm. On the one hand, QPPFeats+inDoc correctly identifies 53% of author assigned keyphrases appearing in the abstract when 15 keyphrases per article are extracted. On the other hand, it can be observed that in -document features degrade the effectiveness of QPPFeats+inDoc in short documents, since 55% of the keyphrases are identified when only QPPFeats are used. Both inDoc and Kea algorithms are able to extract only a maximum of 40% of the keyphrases, which is 15% less than QPPFeats when 15 keyphrases are extracted. Their recall is about half of the QPP features when only 5 keyphrases are extracted.. One important advantage of QPPFeats is that, it is possible to calculate them for phrases not appearing in the original document. In keyphrase generation problem, in contrast to extraction, the algorithm should be able to generate phrases not appearing in the text and should add keyphrase candidates from a prior knowledge such as a background corpus or taxonomy. Expanding the extracted candidate keyphrases is a research topic by itself and is left as a future work. However, in order to demonstrate the fact that QPPFeats can be used in keyphrase generation, we have performed an additional experiment. In the abstracts corpus we have manually added the 55% of the keyphrases that do not occur in their respective abstract to extracted candidate phrase lists and repeated the experiment . In this setting, when top scoring 15 keyphrase candidates are selected, number of correct keyphrases generated is improved by 42.5% and the recall value is increased to 78% with a precision of 20%. The precision value is even higher than the result of inDoc+QPPFeats in the full-text article experiments, i.e. the QPPFeats can extract more keyphrases only by observing the abstracts.. When QPP features are studied individually, it is observed that the two features of CosCentrTod 0 and avgCosTod 0 have the greatest impact on keyphrase extraction. Our experiments suggest that using these two features provides the greatest improvement in keyphrase extraction. Two features , document perturbation (i.e. ranking robustness) and clarity can be successfully used to improve the results both in QPP [21, 22] and keyphrase extraction. Keyphrase extraction algorithms typically score each candidate phrase with a keyphraseness scoring function, and sort the candidates accordingly. This function is implemented either by a supervised or unsupervised learning algorithm. Supervised keyphrase extraction algorithms train a keyphraseness function automatically from observations in a corpus whereas unsupervised keyphrase extraction algorithms depend on scoring functions built on assumptions and observations. Our keyphrase extraction algorithm is a supervised learning algorithm that uses QPP features.. GenEx [6] is a supervised keyphrase extraction algorithm formed of two components: an extractor and a genetic algorithm. The extractor is a text-processing tool controlled by parameters and rules. For example, the aggressiveness of the stemmer and maximum number of the terms allowed in an extracted keyphrase are two parameters o f the extractor. GenEx uses a genetic algorithm and a training set to find the most suitable parameter values for a domain. The population of the genetic algorithm is formed of parameter-sets representing a configuration of the extractor. The fitness function is the precision of the extractor executed with the processed parameter-set/configuration. The output of the genetic algorithm is the set of rules suitable for the corpus domain and genre. GenEx uses frequency and first occurrence position of terms in order to score each phrase's importance.. Kea algorithm uses Naïve Bayes to learn a keyphraseness probability from the in-document features' distribution in the training data. The outline of the Kea algorithm is identical to the outline of the system used in our experiments. Frank et al. [7] report the results of Kea and GenEx [6] to be statistically indifferent. We also use Naïve Bayes in order to learn keyphraseness probabilities of our QPP features. In order to evaluate the QPP features in a systematic manner, we compare our method with Kea [7] algorithm's effectiveness.. Recent research aims to improve the effectiveness of keyphrase extraction by integrating additional features such as those exploiting the structural patterns of a document, syntactic patterns of phrases, semantic knowledge and the relationships between extracted keyphrases. For instance, syntactic features are able to eliminate candidate phrases that are unlikely to be keyphrases (as in the example of the phrase "machine learning introduces " which ends with a verb, and this is uncommon for a keyphrase). Hulth [8] investigates the effectiveness of using part of speech (PoS) tags in keyphrase extraction. She evaluates several candidate phrase extraction strategies with and without PoS tags. Using a rule induction method, PoS tag patterns for keyphrases are learned. Hulth's experiments indicate that keyphrases have common PoS tag patterns. Furthermore, Barker et al. [9] use a chunker to detect noun phrases using simple syntactic patterns, and assign a score to each phrase depending on the tf*idf value of each phrase's head noun. The in-document features are used to evaluate phrases extracted from the document by a noun phrase chunker. Recent work [10] [11] [12] including ours use syntactic filters in finding the candidate keyphrases.. The cohesive ties between keyphrases are exploited in keyphrase extraction by using external knowledge obtained either from thesauri or statistical information extracted from corpora. Turney [13] notes that there should be cohesion between the extracted keyphrases. He measures the pairwise semantic relatedness between two keyphrases by mining the results of a search engine. The cohesion between the keyphrases produced by Kea is reclassified with a second classifier using the cohesion features. This method depends on the output of Kea and in -document features. For extracting keywords, Ercan and Cicekli [14] use the WordNet [15] thesaurus to integrate semantic relationships between phrases and features extracted from the lexical chains of the original document. Thus, their method is not able to handle keyphrases of length greater than one, and is limited to keyword extraction. Nguyen et al. [10] integrate both PoS tags and structural features to the feature set of Kea, where the distinguished structural properties of research articles are important cues for keyphrase extraction. They use the occurrence distribution of phrases with respect to the sections of the processed article. For example, a phrase appearing in the title and/or abstract is more likely to be a keyphrase than a phrase appearing only in the middle of the document. This method is designed specifically for research articles and is not a generic solution.. Mihalcea et al. [11] model the text as a graph where vertices are terms appearing in the given text, and an edge between two terms exists if they co-occur within a distance. Keyphrases are extracted using a social ranking algorithm called TextRank on this network, which is based on the PageRank algorithm [16] . The phrase co-occurrence graph is usually very sparse, especially in short documents where term frequencies are low. Recently, Wan et al. [12] enrich this graph by integrating co-occurrence statistics observed in similar documents, i.e. the nearest neig hbours of the processed text (NN-TextRank). They evaluate their algorithm using news articles that are shorter than research articles, and report improvement over TextRank and a baseline algorithm that scores phrases using tf*idf values. While both QPP features and NN-TextRank use a background corpus in order to handle short documents, these algorithms differ both in methods and features applied.

B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O B I O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Network-On-Chip Design and Synthesis Outlook With the growing complexity in consumer embedded products, new tendencies forecast heterogeneous Multi-Processor Systems-On-Chip (MPSoCs) consisting of complex integrated components communicating with each other at very high-speed rates. Intercommunication requirements of MPSoCs made of hundreds of cores will not be feasible using a single shared bus or a hierarchy of buses due to their poor scalability with system size, their shared bandwidth between all the attached cores and the energy efficiency requirements of final products.To overcome these problems of scalability and complexity, Networks-On-Chip (NoCs) have been proposed as a promising replacement to eliminate many of the overheads of buses and MPSoCs connected by means of general-purpose communication architectures. However, the development of application-specific NoCs for MPSoCs is a complex engineering process that involves the definition of suitable protocols and topologies of switches, and which demands adequate design flows to minimize design time and effort. In fact, the development of suitable high-level design and synthesis tools for NoC-based interconnects is a key element to benefit from NoC-based interconnect design in nanometer scale CMOS technologies.In this article we overview the benefits of state-of-the-art NoCs using a complete NoC synthesis flow, and a detailed scalability analysis of different NoC implementations for the latest nanometer-scale technology nodes. We present NoC-based solutions for on-chip interconnects MPSoCs that illustrate the potential benefits of competitive application-specific NoCs with respect to more regular NoC topologies regarding performance, area and power. Moreover, we show that it is currently Preprint submitted to Elsevier Science 26 November 2007 feasible to synthesize in an automatic way a complete custom NoC interconnect from a high level specification in few hours. Finally, we summarize future research challenges in the area of NoC interconnect design automation. In the last years there has been an increase in computation requirements for embedded systems due to the increasing complexity of new communication and multimedia standards. As a consequence, this has fostered the development of high-performance embedded platforms that can handle the computational requirements of recent complex algorithms, which cannot be executed in traditional embedded mono-processor architectures.. In addition, the continuous time-to-market pressure for consumer embedded devices has made that a design group can no longer perform a complete redesign each time a new product needs to be developed. Due to all these requirements, Multi-Processor System-on-Chip (MPSoC) architectures have become a very attractive solution for the new consumer multimedia embedded market [60] . As a matter of fact, some platforms from the major semiconductor vendors (e.g., Philips Nexperia [41] , TI OMAP [57] or ST Nomadik [51] ) are already available today exemplifying these paradigms in heterogeneous platforms.. Although MPSoCs promise to significantly improve the processing capabilities and versatility of embedded systems, one major problem in their current and future design is the effectiveness of the interconnection mechanisms between the internal components, as the amount of components grows with each new technological node. Bus-based designs are not able to cope with the heterogeneous and demanding communication requirements of MPSoCs. Moreover, as the semiconductor industry reaches deep sub-micron technologies [26] , power density and process variations become critical design concerns for embedded systems as well; thus, predictability in the design of on-chip interconnects is becoming as important as the provided bandwidth. Hence, new paradigms and methodologies that can design power-effective and reliable interconnects for MPSoCs are a must nowadays.. Networks-on-Chip (NoCs) have been suggested as a promising solution to the aforementioned scalability problem of forthcoming MPSoCs [9, 15] . NoCs build on top of the latest evolutions of bus architectures in terms of advanced protocols and topology design, and, by bringing packet-based communication paradigms to the on-chip domain, they address many of the upcoming issues of interconnect fabric design better than buses [3] . For example, wire lengths can be controlled by matching network topology with physical constraints; bandwidth can be boosted simply by increasing the number of links and switches. Furthermore, compared to irregular, bridge-based assemblies of clusters of processing elements, NoCs also help in tackling design complexity and verification issues [23, 10] .. Using NoCs the interconnect structure and wiring complexity can be wellcontrolled. When the interconnect is structured, the number of timing violations that occur during the physical design (floorplanning and wire routing) phase are minimal. Such design predictability is critical for todays MPSoCs to achieve timing closure. It leads to faster design cycle, reduction in the number of design re-spins and faster time-to-market. As the wire delay as a fraction of gate delay is increasing with each technological generation, having shorter wires is even more important for future MPSoCs. Early works on NoC topology design assumed that using regular topologies, such as mesh, like those that have been used in macro-networks, would lead to regular and predictable layouts [20, 10] . While this may be true for designs with homogeneous processing cores and memories, it is not true for most MPSoCs as they are typically composed of heterogeneous cores and regular topologies result in poor performance, with large power and area overhead. This is due to the fact that the core sizes of the MPSoC are highly non-uniform and the floorplan of the design does not match the regular, tile-based floorplan of standard topologies [10] . Moreover, for most state-of-the-art MPSoCs (like the Cell-Playstation III [56] , Philips Nexperia [37] or TI OMAP [57] ) the system is designed with static (or semi-static) mapping of tasks to processors and hardware cores, and hence the communication traffic characteristics of the MPSoC can be obtained statically. Thus, an application-specific NoC with a custom topology, which satisfies the design objectives and constraints, is critical to have efficient on-chip interconnects for MPSoCs.. When designing an efficient NoC architecture, satisfying the application performance constraints is a complex process. The design issues span several abstraction levels, ranging from the high-level application modeling to the physical layout-level implementation. Some of the most important phases in designing NoCs include modeling the application traffic characteristics, synthesizing the topology or structure of the network, setting various design parameters (such as frequency of operation or link width), generating the Register Transfer Level (RTL) code for the network components and performing the physical design. To handle the design complexity and meet the tight time-to-market constraints, it is important to automate most of these NoC design phases. To achieve design closure, the different phases should also be integrated in a seamless manner, which is not an easy challenge and opens many commercial opportunities. In fact, three companies already exist in this space that have started bringing the NoC technology to commercial product, namely, Arteris [39] , Silistix [49] and iNOCs [21] , and more start-ups are likely to appear in the growing market of MPSoC on-chip interconnect design. This paper tries to provide an overview of the currently available NoC-based interconnects for next-generations of MPSoC platforms. Thus, to take into account as many key effects as possible in our studies of NoC interconnects, we establish a flow in this work that takes our MPSoC test platforms down to placed&routed layouts. This flow allows us to derive final frequency, area and power figures for the NoC blocks to perform complete studies of different overall NoC interconnects. In our analyses and study of on-chip interconnects we cover NoCs implemented with the proposed design flow using three different technology libraries (130, 90 and 65nm), such that we can provide conclusions for a very representative part of the design spectrum. Finally, according to our conclusions we suggest a number of future research challenges in the area of NoC technologies and interconnect design automation for forthcoming MPSoC embedded platforms.. The remainder of the paper is structured in the following way. In Section 2, we present an overview of state-of-the-art in the area of on-chip interconnect architectures and NoC design methods for MPSoCs. Then, Section 3 presents the architectural foundations of NoCs. Next, in Section 4 we describe the proposed design flow and exploration front-end to build reliable NoC-based interconnects for MPSoCs and Section 5 covers in detail the backend process necessary for suitable NoC synthesis in latest process technologies. After this, in Section 6 we review the characteristics of state-of-the-art NoCs using the proposed NoC synthesis flow, and perform a complete scalability analysis of different NoC implementations for several real-life MPSoC case studies using the latest nanometer-scale technology nodes. Then, in Section 7 we present an overview of possible future research challenges that still need to be addressed in NoC interconnects to be valid for future MPSoC designs and 3D integration technologies. Finally, Section 8 summarizes the major conclusions obtained from this work. Emerging consumer applications demand a very high-level of performance in the next generation of embedded devices. Therefore, new techniques and interconnection mechanisms that can provide solutions for an efficient design of these complex forthcoming embedded architectures are greatly needed. NoCs have emerged as a promising structured way of realizing interconnections on silicon, overcoming the limitations of bus-based solutions. In this paper, we have performed a thorough study of the current state-of-the-art of NoC implementations using a design flow targeting the new trends imposed by deep sub-micron manufacturing processes. Also, we have presented a comparative analysis of different NoC fabrics ranging from regular topologies to highlytuned custom NoCs.. In this regard, we have illustrated that to have fewer design re-spins and faster time-to-market, design flows for NoC interconnects need to integrate the architectural models with back-end physical design models, thereby bridging a big design gap in NoC synthesis and creating on-chip interconnects free from deadlocks. Moreover, to handle the wiring complexity issue in NoC synthesis, accurate estimations of the interconnect delay and power consumption of the basic building blocks of NoCs early in the design phase are key to produce suitable interconnects for latest MPSoCs. We have also shown that custom NoC topology design, where a NoC is tailored to fit the target application, has noticeable potential benefits with respect to regular topologies, such as, meshbased NoCs. All in all, the current NoC architectural blocks and design flows for custom NoC topology design have reached a level of maturity comparable to traditional bus-based on-chip interconnects. In fact, the presented design flow is currently able to synthesize a complete NoC-based custom interconnect for a certain MPSoC architecture, starting from a high level specification, in few hours.. Nevertheless, several research directions still exist to make feasible efficient designs of NoC interconnects for new nano-scale devices and technologies. First, efficient support at the NoCs level to handle blocks of MPSoCs working at different operating frequencies or GALS is still an open question. Also, mechanisms for NoC partitioning to provide QoS for new downloaded applications is a very challenging problem. Additionally, the possible benefits of applying runtime dynamic control to NoCs (e.g., dynamic routing schemes) to improve statically partitioned MPSoCs at compile time is an open question. All these extensions to current NoC designs would require changes in both the synthesis algorithm and architectural implementations. Finally, a novel and very interesting research avenue is the design of 3D NoCs, which target next-generation stacked chips. In this regard, according to the yield and speed of vertical interconnects, many different NoC architectures and topologies can be envisioned, as well as the possible inclusion of various fault tolerant schemes, and the definition of suitable choices for each case are still to be explored. Moreover, the CAD support in this case would become even more critical and we expect a large set of new research challenges for NoC design in this area. A significant number of different communication fabrics have been described in the literature. The different versions of the ARM Advanced Microcontroller Bus Architecture (AMBA) [6] , including the latest AXI [5] standard, and the ST Microelectronics STBus [34] and Sonics MicroNetworks [59] are examples of advanced buses that attempt to overcome the limitations of classical shared bus architectures and bridge-based assemblies of clusters of processing elements by different methods. For instance, multiple STBus channels can be deployed, leading to crossbars in the extreme solution. In fact, most of these bus solutions are getting very similar in their parallel multi-channel architectures for shared on-chip communication to the packet-based interconnects promoted by the NoC paradigm. Thus, making the differentiation between both paradigms for on-chip interconnects, traditionally differentiated, become almost inexistent.. Up to today several researchers have motivated the need for NoC-based designs as a way to tackle design complexity issues [17, 15, 28, 9, 23] . Research on NoC architectures has fostered the proposal of different flow control protocols [10, 43, 17] , Quality of Service (QoS) provisions [16, 13, 58] , and asynchronous implementations of NoCs [12, 47] . Moreover, several start-ups already exist today that are bringing NoC technologies into commercial products [39, 49, 21] , thus, the application of NoCs into real-life MPSoCs is already becoming a reality.. In addition, various works in the literature exist that explore the implications of the NoC paradigm at different design levels trying to propose efficient synthesis methods for NoC-based interconnects and to provide coherent comparisons with bus-based MPSoCs [20, 35, 3] . NoC layouts for Application Specific Integrated Circuits (ASICs) are presented in [44, 4, 45, 2] , a test chip is shown in [29] , and different implementations of NoCs on Field Programmable Gate Arrays (FPGAs) are provided by [61, 32] .. Moreover, the design of application-specific NoC architectures for known communication patterns has been addressed in [42, 19, 18, 50, 1] . Also, several researchers have developed tool chains for designing application-specific NoCs [17, 27] , while a complete Computer-Aided Design (CAD) tool for NoC instantiation and optimization for both regular and custom topologies can be found in [11] . Synthesis and layout results for the xpipes library of component blocks that we leverage upon for our study of NoC architectures are shown in [4] .

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O