Integrating TwoTowers and GreatSPN through a Compact Net Semantics Stochastic process algebras (SPAs) and stochastic Petri nets (SPNs) are two well known formal methods for the functional and performance modeling and analysis of computer, communication and software systems. Starting from the mappings from process algebras to Petri nets proposed in the literature to provide a truly concurrent semantic framework to concurrent programming languages, in this paper we define a new SPN semantics for SPAs in order to facilitate the integration and the cross fertilization between the two formalisms. We then prove that our net semantics is correct via a retrievability result. Afterwards, we demonstrate that it improves on the previously proposed net semantics with respect to the size of the resulting SPNs and on the standard interleaving semantics because of the detection of system symmetries. Furthermore, we illustrate its usefulness by showing how to reinterpret at the SPA level the results efficiently obtainable at the SPN level. Finally, we describe the implementation of our net semantics that has been realized to integrate the EMPA gr based software tool TwoTowers with the GSPN based software tool GreatSPN. The complexity of the modern computer, communication and software systems calls for the development of suitable formal description techniques for their modeling and analysis. In the literature, both process algebras and Petri nets have received much attention, as witnessed by their extensive investigation in the last four decades (see, e.g., [24, 27] and the references therein). Process algebras and Petri nets have been studied not only in isolation but also jointly, the purpose being that of providing a truly concurrent semantic framework to concurrent programming languages. This has been accomplished by means of semantic mappings from process algebras to Petri nets [19, 16, 25, 9] .. More recently, it has been recognized that the net semantics for process algebras have an applicative relevance, as they facilitate the integration of the two formalisms making possible the exchange of modeling capabilities and analysis techniques between them [25, 28, 4, 3] . This is especially important since the two formalisms are characterized by complementary strengths. On the one hand, process algebras offer a compositional linguistic support to system modeling together with congruences that are useful for compositionally reducing the state space underlying the process terms before analyzing them [24] . On the other hand, Petri nets provide a truly concurrent framework equipped with structural analysis techniques that avoid the construction of the underlying state space [27] . Such considerations are valid not only from the functional point of view, but also from the performance point of view [22, 20, 8, 11] . In particular, they have been taken into account in [3] , where a formal approach to the modeling and analysis of complex systems has been proposed, which is based on stochastic process algebras (SPAs) and stochastic Petri nets (SPNs). The objective of the approach is to allow both functional and performance aspects of systems to be considered from the early stages of their design, so that malfunctionings and inefficiency can be detected from the very beginning, thereby avoiding project cost increases that would be caused by their late discovery. In order to be effective, this integrated approach must be implemented through a software tool that should assist the performance modeler. Additionally, such a software tool should made transparent to the performance modeler the transformation of SPA models into SPN models and the reinterpretation at the SPA level of the results of the analysis conducted at the SPN level.. Two fundamental techniques are known in the literature for defining the semantics for a language: the denotational one and the operational one [30] . In this paper we concentrate on the latter technique, i.e. we consider only operational SPN semantics for SPAs. The reason is that operational mappings define abstract machines, whose implementation is straightforward in a software tool, that generate SPN models in an incremental way. The operational mappings from SPAs to SPNs can be evaluated on the basis of their com-pactness. By compactness we mean the size of the resulting SPNs in terms of number of places and transitions. Obtaining a more compact SPN may lead to a more efficient functional and performance analysis of the modeled system, since the underlying linear equation system on which the analysis is based can be smaller.. In the literature, two different approaches to operational Petri net semantics for process algebras have been defined. In the location oriented approach, the Petri net corresponding to a process term contains a place for every position of the sequential subterms with respect to the static operators (such as hiding, relabeling, and parallel composition). As a consequence, the resulting nets turn out to be 1-safe, i.e. every place contains at most one token at any time.. All the information about the syntactical structure of the terms is encoded within the places, so that the relationships among the sequential terms are smoothly preserved. This is exploited to define the net transitions by means of inductive rules similar to those for the interleaving semantics for the process algebra, as shown in [16, 25] .. In the label oriented approach, instead, each net place corresponds to a sequential subterm independently of its position with respect to the static operators. As a consequence, the instances of the same sequential term occurring in different positions with respect to the static operators can be represented by multiple tokens within the same place. This is possible because the syntactical structure of the process terms with respect to the static operators is no longer fully retained within the net places associated with the sequential subterms. Therefore, the correspondence between the inductive rules for generating the net transitions and the inductive rules of the interleaving semantics for the process algebra no longer holds. In this approach the net transitions are defined by axioms. For instance, in [9] five axioms are employed to derive the net transitions, with the resulting nets including places with inhibitor arcs to model scope extrusion and unguarded choice. As another example, in [4, 3] only one axiom is employed, the price to be paid being the introduction of places with inhibitor and contextual arcs in which information about the syntactical structure is kept.. In terms of compactness, the label oriented approach is advantageous since the resulting nets are not necessarily 1-safe and therefore they can be smaller than those obtained with the location oriented approach. Furthermore some terms, for which an infinite net is generated in the location oriented approach, give rise to a finite net in the label oriented approach [10] . Given the applicative relevance of the integration of SPAs and SPNs, the label oriented approach seems to be the approach of choice to realize such an integration due to its compactness.. The weakness of this approach is that additional places with inhibitor and contextual arcs are employed to handle static operators. Actually, in [4, 3] it has been proved that these places with the related arcs can be removed a posteriori; this makes the nets simpler but requires an additional computational step. The purpose of this paper 2 is to further elaborate on the label oriented SPN semantics of [4, 3] by avoiding the introduction of additional places with inhibitor and contextual arcs altogether. The novel idea to achieve that is to suitably decorate the actions within the sequential terms associated with the net places in order to keep track of the occurrences of the static operators, without falling in the excess of information of the location oriented approach.. The new label oriented net semantics is developed by focusing on EMPA gr (Extended Markovian Process Algebra with generative-reactive synchronizations) [8] and on GSPNs (Generalized Stochastic Petri Nets) [1] . The reason is twofold. First, the structure of EMPA gr actions has been strongly influenced by the structure of GSPN transitions, so the mapping from EMPA gr terms to GSPNs should be natural. Second, analysis tools -TwoTowers [3] and GreatSPN [13] -have been developed for both formalisms, so that the integration of the two formalisms can be realized in practice. It is worth noting that, since the semantics is intended to be implemented in a software tool that should assist the performance modeler, all the details of the transformation are completely transparent. This paper is organized as follows. In Sect. 2 we recall EMPA gr and GSPNs. In Sect. 3 we define an improved, label oriented GSPN semantics for EMPA gr and we prove its correctness by showing that the interleaving semantics for EMPA gr can be retrieved from it. In Sect. 4 we discuss the compactness of the generated GSPNs both at the net level, with respect to the previously proposed net semantics, and at the state space level, by showing that some system symmetries are captured; we also exhibit a sufficient condition for the finiteness of the generated GSPNs and we investigate the reinterpretability of the analysis results. In Sect. 5 we describe how the novel label oriented GSPN semantics for EMPA gr is implemented to integrate TwoTowers with GreatSPN. In Sect. 6 we present an example in which we exploit the features of the net semantics to measure the performance of a random polling system when varying its parameters. Finally, Sect. 7 concludes the paper with some remarks on related and future work. In this paper we have presented and proved correct a new label oriented GSPN semantics for EMPA gr , which improves all the net semantics previously published in the literature with respect to the size of the generated nets and allows functional and performance properties efficiently derived at the GSPN level via structural techniques to be reinterpreted at the EMPA gr level. Because of its practical advantages, such a label oriented GSPN semantics has been implemented to integrate the EMPA gr based software tool TwoTowers with the GSPN based software tool GreatSPN, in order to create a tool for the formal modeling and analysis of complex systems that benefits from the complementary strengths of both formalisms.. Related Work. A comparison with the net semantics for process algebras previously proposed in the literature has been made throughout the paper. As we have seen, one of the advantages of our new semantics -the compactness of the resulting GSPNs -allows system symmetries to be captured and exploited at analysis time.. The detection of symmetries for state space reduction is not new in the field of SPAs. In [18] a reduced state space underlying a PEPA [22] model is obtained by building equivalence classes of states and by considering one element for each class only. Equivalence classes of states are computed syntactically by looking at the structure of the terms and by considering as equivalent those states which differ in the position of the components within a parallel composition. For example, if we consider the term E S E, its possible derivatives E S E and E S E are considered equivalent and therefore belong to the same equivalence class. The reduction in the size of the state space is not optimal -it is still possible to find states which are Markovian bisimulation equivalent but do not belong to the same equivalence class -but it is stronger than ours since it works for synchronized replicas whose corresponding GSPNs are kept distinct in our framework. In this respect, it is worth observing that the advantage of having a GSPN representation of an EMPA gr term is not limited to state space reduction only. In fact, structural techniques for functional verification and performance evaluation like those mentioned in Sect. 4.4 can be applied to the GSPN representation of an EMPA gr specification, thereby avoiding the construction of the underlying state space. Another approach to state space reduction is discussed in [21] . Here a new operator is introduced for representing sets of identical processes, all synchronizing on the same sets of actions (possibly the empty set). New semantic rules have been defined, which are consistent with those for the traditional parallel composition operator but avoid the representation of all possible interleavings. An informal net semantics which generates k-safe nets is also discussed. Finally, we mention the work in [17] where an attempt is made to adapt for PEPA the structural analysis techniques typical of Petri nets. This is achieved thanks to a matrix representation of PEPA components that is reminiscent of incidence matrices of Petri nets.. Future Work. The GSPN model of the random polling system we have shown in Sect. 6 is very compact since the net semantics captures all the intrinsic symmetries of the system. In order to be able to capture a higher number of symmetries for more complex systems, we may think of changing the underlying model of our semantics in favor of more appropriate models for which reduced state space construction methods have been already proposed. For instance, we may think of mapping EMPA gr terms onto stochastic well formed nets (SWNs) [12] or stochastic activity networks (SANs) [23] . As an example, in a random polling system there might be a customer that is faster than the others. In such a case the customers are still equivalent from a purely functional point of view, but the GSPN generated by applying our net semantics to the EMPA gr specification of the system contains a distinct subnet for the fast customer because of the different rate values. The same would not happen e.g. in a SWN model since there it is possible to define transition rates depending on the identity of the tokens, i.e. on the identity of the customers. In this section we recall some concepts about the two specific instances of SPAs and SPNs, respectively, we shall consider in the rest of the paper. In Sect. 2.1 we introduce EMPA gr while in Sect. 2.2 we present GSPNs. The reader who is already familiar with EMPA gr or GSPNs can safely skip the related section.

O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Exploiting hierarchical clustering in evaluating multidimensional aggregation queries Multidimensional aggregation queries constitute the single most important class of queries for data warehousing applications and decision support systems. The bottleneck in the evaluation of these queries is the join of the usually huge fact table with the restricted dimension tables (star-join). Recently, a multidimensional hierarchical clustering schema for star schemas is suggested. Subsequently, query evaluation plans for multidimensional queries appeared that essentially implement a star join as a multidimensional range restriction.We present a number of transformations for such plans. The transformations place grouping/aggregation operations before joins and safely prune aggregated tuples. They can be applied at no or minimal extra I/O cost. We show how these transformations can be used to construct a new evaluation plan for grouping/aggregation queries over multidimensional hierarchically clustered schemas. The new plan improves previous results by grouping and aggregating tuples and by excluding aggregated tuples from further consideration at an early stage of the computation of a query. Multidimensional aggregation queries constitute the single most important class of queries for data warehousing applications and decision support systems. Warehouse data are usually modeled using a multidimensional data model. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DOLAP'03, November 7, 2003 Most data warehouses implemented following the relational approach use the star schema [9, 2] to represent the multidimensional data model where a fact table is surrounded by a number of dimension tables. The dimension tables represent hierarchies specifying different aggregation levels i.e., different granularities of viewing data. Conceptually, a multidimensional aggregation query restricts dimension tables, joins the resulting tables with the fact table and then groups and aggregates the joined table.. The bottleneck in the evaluation of multidimensional aggregation queries is the join of the usually huge fact table with the restricted dimension tables (called star-join). Bitmapped indexes [12] and variations have been the most popular choice for dealing with star-joins and the focus of extensive research [13, 14, 1] . However, in many cases multidimensional access methods outperform bit-mapped indexing methods [14] . This remark suggested that multidimensional access methods might be better suited for data warehousing applications. Multidimensional aggregation queries restrict hierarchy attributes over multiple dimensions. Multidimensional access methods [5] cluster data with respect to multiple dimensions, and are used in spatial DBMS to efficiently compute multidimensional range queries. Clustering a onedimensional hierarchy is a well known technique [19] . Subsequently, a multidimensional hierarchically clustering access method for the fact table that combined both techniques was introduced in [11] in order to speed up multidimensional aggregation queries.. The new star schema organization triggered research in both directions: (a) finding new physical operators and algorithms for their implementation that would take advantage of the new data structures, and (b) finding efficient query evaluation plans involving the new physical operators that could be integrated in query optimizers.. With respect to the first direction, the Tetris algorithm [11] is a generalization of a multidimensional range query algorithm that efficiently combines sort operations with the evaluation of multidimensional restrictions. It operates with any multidimensional access method that creates a disjoint partitioning of the multidimensional space. Compared to the access methods of commercial systems for grouping/aggregation queries it shows significant speedups, important temporary storage requirement reduction for the sorting process, and multiple times faster production of the first results of a sort operation [11, 10] .. With respect to the second direction, the query evaluation plans presented in [15, 8] consider that the fact table is hierarchically clustered with respect to multiple dimensions.. General transformation rules in the presence of dependencies were investigated in [16] . Interestingly, it became immediately apparent that prior work on partially or totally pushing group by operations past one or more join operations (also called eager aggregation transformation) [17, 3, 18, 7, 4] could be applied to these plans to partially group and aggregate tuples that are selected from the fact table. This transformation is not possible in traditional star schemas where no information about the hierarchies is encoded in the fact table. Contribution In this paper we present a new query evaluation plan for multidimensional aggregation queries that fully exploits the properties of a multidimensional hierarchically clustered star schema. This plan exploits: (a) multidimensional range query algorithms to implement expensive star-join operations as multidimensional range restrictions on the fact table followed by merge-join operations with restrictions of the dimension tables, and (b) the sorting feature of the Tetris algorithm to group selected fact table tuples on compound surrogate attributes that encode hierarchy information. We present a number of transformations that allow placing early grouping/aggregation operations at different levels of the processing of a query before the joins of fact table tuples with the dimension tables. These operations can be performed at no or minimal extra I/O cost by exploiting the sort order of intermediate result relations. We also present transformations that allow a safe pruning of aggregated tuples. We show how these transformations can be used to construct the new query evaluation plan. These contributions importantly improve previous results on evaluating multidimensional aggregation queries [15, 8] by grouping and aggregating tuples and by excluding aggregated tuples from further consideration at an early stage of the processing of a query. Recently, a new star schema for multidimensional databases is suggested that clusters hierarchically the fact table with respect to multiple dimensions. Evaluation plans for multidimensional aggregation queries over this schema implement the expensive star-join operation of traditional evaluation plans as a multidimensional range restriction on the fact table followed by joins with the dimension tables. The multidimensional hierarchical clustering of the fact table allows eager aggregation transformations to be applied to these plans which were not possible in a traditional evaluation of these queries. We have presented a number of transformations that allow grouping/aggregation operations to be partially or totally pushed past join operations and transformations that allow the pruning of aggregated tuples. We have shown how these transformations can be exploited in constructing a new query evaluation plan for multidimensional aggregation queries over multidimentional hierarchically clustered star schemas. The new plan improves previous ones since it can perform a grouping and aggregation operation before every join operation, if necessary, and excludes aggregated tuples from further consideration at an early stage of the computation of a query.. We believe that the evaluation plan for multidimentional aggregation queries presented here should be incorporated to query optimizers for OLAP queries. The query optimizer should also decide, based on a cost model and statistical information, about the order of joining the (restricted) dimension tables with the tuples derived from the fact table as this is determined by the Tetris operator. There are three, possibly rival, factors that affect this decision: (a) the performance of the Tetris operator for different orderings of the compound surrogate attributes of the joining dimensions, (b) the number and the size of the imported attributes of each joining dimension (if any), and (c) the number of aggregated tuples an produced by an early grouping/aggregation operation. Step 1

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An iterative multimodal framework for the transcription of handwritten historical documents The transcription of historical documents is one of the most interesting tasks in which Handwritten Text Recognition can be applied, due to its interest in humanities research. One alternative for transcribing the ancient manuscripts is the use of speech dictation by using Automatic Speech Recognition techniques. In the two alternatives similar models (Hidden Markov Models and n-grams) and decoding processes (Viterbi decoding) are employed, which allows a possible combination of the two modalities with little difficulties. In this work, we explore the possibility of using recognition results of one modality to restrict the decoding process of the other modality, and apply this process iteratively. Results of these multimodal iterative alternatives are significantly better than the baseline uni-modal systems and better than the non-iterative alternatives. In the last years, many on-line archives and digital libraries are publishing large quantities of digitised legacy documents. These documents must be transcribed into an appropriate textual electronic format in order to allow text-based search of their contents and provide historians and other researchers new ways of indexing, consulting and querying their contents. However, the vast majority of these documents (hundreds of terabytes of digital image data) remain waiting to be transcribed into a textual electronic format. Therefore, manual transcription of these documents is an important task for making available the contents of digital libraries.. These transcriptions are usually carried out by experts in paleography, who are specialised in reading ancient scripts. These scripts are characterised by different handwritten/printed styles from diverse places and time periods. The time that takes for an expert to make a transcription of one of these documents depends on their skills and experience. Most paleographs agree that each page needs several hours to be transcribed.. In this context, Handwritten Text Recognition (HTR) (Toselli et al., 2004) has become an important research topic. HTR tries to obtain the word sequence contained in the image of a handwritten text line. This process needs a previous detection of lines of text in an image, as well as some preprocessing steps to make the handwritten text more regular. The final result is a sequence of words (transcription) of the text line, that may contain errors. When the rate of errors of the transcription is low enough, HTR can be a very useful tool to speed up the transcription of handwritten text documents.. However, when consulting paleographs on the most comfortable method to transcribe a handwritten text document, many of them claim that a dictation of the words is the best option. Consequently, Automatic Speech Recognition (ASR) systems are an important alternative to HTR systems. In addition, the current state-of-the-art ASR and HTR systems share many features: Hidden Markov Models (HMM) (Jelinek, 1998; Rabiner, 1989 ) are used to model the basic elements of the signal (sounds for speech, strokes for handwritten text) and n-grams language models (LM) are used to model word sequences (Jelinek, 1998) . From this viewpoint, HTR systems fit in the Natural Language Processing paradigm. Therefore, many features that are usual to ASR systems (such as the use of training data for HMM and n-grams) are common to HTR systems as well.. The similarities between the two types of systems make possible to combine them easily into a multimodal system that may obtain a more reliable final hypothesis, since two different data sources (handwritten text and speech) can be used. In fact, previous attemps in combining handwritten input and speech input have been done (Liu and Soong, 2009 ), but most of them focus on the use of on-line handwritten text. In a previous work (Alabau et al., 2011) a first attempt of combining off-line HTR and ASR systems showed promising results. The method consisted basically of restricting the ASR decoding process based on the results of the HTR decoding. In this work we extend the process described in Alabau et al. (2011) towards two different directions: we explore the effect of using the different modalities (HTR and ASR) as starting modality, and we study the iterative use of the process.. The paper is organised as follows: Section 2 describes the fundamentals of HTR and ASR systems, Section 3 explains the use of the HTR decoding to improve the ASR recognition, Section 4 summarises the experimental set-up, Section 5 shows the results, and Section 6 provides the main conclusions and future work lines in this field. In this article we have tackled the transcription of handwritten historical documents from a multimodal (HTR and ASR) and iterative perspective. The multimodal system can be compared against two uni-modal baseline systems and the iterative approach can be compared against the non-iterative multimodal system.. The multimodal approach shows a significant improvement in results with respect to the baseline uni-modal system. The adoption of the iterative approach shows a greater improvement in the results with respect to the baseline uni-modal systems, although not always significantly better than multimodal non-iterative results.. The influence of the initial modality in behaviour of the multimodal iterative approach was studied for the two modalities. Results show that starting from ASR allows for a better final performance. In all cases of the iterative process, HTR results present a poorer result that their ASR counterparts, but their decoding parameters seem not as optimal as the decoding parameters of the ASR systems.. Although errors are significantly less than those of baseline systems and time response is appropriate for the transcription task, more work can be done to improve these two aspects and obtain a better performance in a real system. On the one hand, HTR decoding needs a better tuning for the recognition parameters, that can be obtained in a similar way to those obtained for the ASR decoding; the improvement in the HTR decoding could cause a positive impact in the final performance of the iterative systems, both in terms of WER and convergence time. On the other hand, an integrated decoding of HTR and ASR is a suggestive alternative to the iterative paradigm that can reduce dramatically the final time and obtain better quality in the automatic transcriptions. This section is devoted to analyse the experimental results of the methods proposed in Section 3 and the iterative process described in Section 3.2. The results will be compared against a baseline result which uses only one modality and in a non-iterative fashion.

O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


First-Order Logic on Higher-Order Nested Pushdown Trees We introduce a new hierarchy of higher-order nested pushdown trees generalising Alur et al.'s concept of nested pushdown trees. Nested pushdown trees are useful representations of control flows in the verification of programs with recursive calls of first-order functions. Higher-order nested pushdown trees are expansions of unfoldings of graphs generated by higher-order pushdown systems. Moreover, the class of nested pushdown trees of level n is uniformly first-order interpretable in the class of collapsible pushdown graphs of level n + 1. The relationship between the class of higher-order pushdown graphs and the class of collapsible higher-order pushdown graphs is not very well understood. We hope that the further study of the nested pushdown tree hierarchy leads to a better understanding of these two hierarchies. In this article, we are concerned with the first-order model checking problem on higher-order nested pushdown trees. We show that the first-order model checking on the first two levels of this hierarchy is decidable. Moreover, we obtain an alternating 2-EXPTIME algorithm for the class of nested pushdown trees of level 1. The proof technique involves a pseudo-local analysis of strategies in the Ehrenfeucht-Fraïssé games on two identical copies of a nested pushdown tree. Ordinary locality arguments in the spirit of Gaifman's lemma do not apply here because nested pushdown trees tend to have small diameters. We introduce the notion of relevant ancestors which provide a sufficient description of the FO k -type of each element in a higher-order nested pushdown tree. The local analysis of these ancestors allows us to prove the existence of restricted winning strategies in the Ehrenfeucht-Fraïssé game. These strategies are then used to create a first-order model checking algorithm. During the last decade, different generalizations of pushdown systems have gained attention in the field of software verification and model checking. Knapik et al. [2002] showed that Higher-order pushdown systems, first defined by Maslov [1974 Maslov [ , 1976 , generate the same class of trees as safe higher-order recursion schemes. Safety is a syntactic condition concerning the order of the output compared to the order of the inputs of higher-order recursion schemes. A higher-order pushdown system is a pushdown system that uses a nested stack structure instead of an ordinary stack. This means that a level 2 pushdown system (2-PS) uses a stack of stacks, a level 3 pushdown system (3-PS) uses a stack of stacks of stacks, etc. Hague et al. [2008] defined the class of collapsible pushdown systems by adding a new stack operation called collapse. They proved that the trees generated by level l collapsible pushdown systems coincide with the trees generated by level l recursion schemes. The exact relationship between the higher-order pushdown hierarchy and the collapsible pushdown hierarchy remains an open problem. It is not known whether the trees generated by safe recursion schemes are a proper subclass of the trees generated by all recursion schemes. 1 Due to the correspondence of recursion schemes and pushdown trees, this question can be equivalently formulated as follows: is there some collapsible pushdown system that generates a tree which is not generated by any higher-order pushdown system? Also from a model-theoretic perspective these hierarchies are interesting classes. The graphs generated by higher-order pushdown systems are exactly the graphs in the Caucal-hierarchy [Carayol and Wöhrle 2003 ]. Thus, they are one of the largest known classes with decidable monadic second-order theories. In contrast, Broadbent [2012] showed that the first-order theories of graphs generated by level 3 collapsible pushdown systems are in general undecidable (level 2 collapsible pushdown graphs have undecidable monadic second-order theories but decidable first-order theories [Hague et al. 2008; Kartzow 2010] ). Thus, the collapse operation induces a drastic change with respect to classical decidability issues.. Furthermore, collapsible pushdown graphs have decidable modal μ-calculus theories [Hague et al. 2008] . In fact, the class of collapsible pushdown graphs and the class of nested pushdown trees are the only known natural classes with decidable modal μ-calculus theories but undecidable monadic second-order theories.. Further study of the higher-order pushdown hierarchy and the collapsible pushdown hierarchy is necessary for a better understanding of these results. It may also reveal an answer to the question whether safety implies a semantical restriction for recursion schemes.. In this article we introduce the hierarchy of higher-order nested pushdown trees. This is a new hierarchy between the hierarchy of higher-order pushdown trees and that of collapsible pushdown graphs. We hope that its study reveals more insights into the structure of these hierarchies.. Nested pushdown trees were first introduced by Alur et al. [2006] . These are trees generated by pushdown systems (of level 1) enriched by a new jump relation that connects each push operation with the corresponding pop operations. They introduced these trees in order to verify specifications concerning pre/postconditions on function calls/returns in recursive first-order programs. "Ordinary" pushdown trees offer suitable representations of control flows of recursive first-order functions. Since these trees have decidable monadic second-order theories [Muller and Schupp 1985] , one can use these representations fruitfully for verification purposes. But monadic second-order logic does not provide the expressive power necessary for defining the position before and after the call of a certain function in such a pushdown tree. Alur et al.'s new jump relation makes these pairs of positions definable by a quantifier-free formula. Unfortunately, this new relation turns the monadic second-order theories undecidable. But they showed that modal μ-calculus model checking is still decidable on the class of nested pushdown trees. Thus, nested pushdown trees form a suitable representation for control flows of first-order recursive programs for the verification of modal μ-calculus definable properties of the control flows including pre/postconditions on function calls/returns.. Of course, the idea of making corresponding push and pop operations visible is not restricted to pushdown systems of level 1. We define a level n nested pushdown tree (n-NPT) to be a tree generated by a level n pushdown system (without collapse!) expanded by a jump relation that connects every push of level n with the corresponding pop operations (of level n).. This new hierarchy contains by definition expansions of higher-order pushdown trees. Moreover, we show that the class of n-NPT is uniformly first-order interpretable in the class of level n + 1 collapsible pushdown graphs.. We then study first-order model checking on the first two levels of this new hierarchy. In particular, we provide a 2-EXPTIME alternating Turing machine deciding the model checking problem for the class of 1-NPT. We already proved the same complexity bound in [Kartzow 2009 ]. Here, we reprove the statement with a different technical approach that generalizes to the higher levels of the nested pushdown tree hierarchy.. Outline. Section 2 contains some basic definitions concerning first-order logic and higher-order pushdown systems. Moreover, we recall the basics of Ehrenfeucht-Fraïssé games and explain how the analysis of strategies in these games can be used to derive first-order model checking algorithms on certain classes of structures. In Section 3, we then introduce the hierarchy of nested pushdown trees. We relate this hierarchy to the hierarchies of pushdown trees and of collapsible pushdown graphs. From that point on, we only focus on the first-order model checking problem for the first two levels of the nested pushdown tree hierarchy. In Section 3.1, we explain the rough picture how the ideas of Section 2.1 lead to a model checking algorithm for this class. We then give an outline of Sections 4-7, which provide the details of the correctness proof for the model checking algorithm presented in Section 7. Finally, we give some concluding remarks and point to open problems in Section 8. In this article we extended the notion of a nested pushdown tree and developed the hierarchy of higher-order nested pushdown trees. These are higher-order pushdown trees enriched by a jump relation that makes corresponding push and pop operations (of the highest level) visible. This new hierarchy is an intermediate step between the hierarchy of pushdown trees and the hierarchy of collapsible pushdown graphs in the sense that it contains expansions of higher-order pushdown trees and its n-th level is uniformly first-order interpretable in the class of collapsible pushdown graphs of level n + 1. We hope that further study of this hierarchy helps to clarify the relationship between the hierarchies defined by higher-order pushdown systems and by collapsible pushdown systems. We have shown the decidability of the first-order model checking on the first two levels of the nested pushdown tree hierarchy (note that we are at the border to undecidability because first-order logic on the class of all level 3 collapsible pushdown graphs [Broadbent 2012 ] is undecidable). The algorithm is obtained from the analysis of restricted strategies in Ehrenfeucht-Fraïssé games on nested pushdown trees of level 2. The game analysis gets tractable due to the theory of relevant ancestors in combination with shrinking constructions for level 2 pushdown systems. It is unknown whether this approach extends to higher levels. The theory of relevant ancestors generalizes to all levels of the hierarchy and provides an understanding of the Ehrenfeucht-Fraïssé games. Unfortunately, we do not have any kind of shrinking constructions for pushdown systems of level 3 or higher. The development of such shrinking construction may yield the necessary bridge between the theory of relevant ancestors of n-NPT and the dynamic-small-witness property needed for the development of a model checking algorithm. Furthermore, better shrinking construction may imply elementary complexity bounds for the FO model checking on 2-NPT. Another open question concerns the modal μ-calculus model checking on this new hierarchy. Note that the interpretation of n-NPT in collapsible pushdown graphs of level n + 1 is almost modal but for the reversal of the jump edges in comparison to the collapse edges used for their simulation. We conjecture that modal μ-calculus is decidable on the whole hierarchy of nested pushdown trees. Moreover, it is open how the nested pushdown tree hierarchy relates exactly to the (collapsible) higher-order pushdown hierarchies. We conjecture that the relationships described in Section 3 are optimal but we lack proofs. For instance, it is open whether n-NPT can be interpreted in level n collapsible pushdown graphs or vice versa (we conjecture the answer is no).

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Cluster miss prediction for instruction caches in embedded networking applications In this paper, we describe a new method of instruction prefetching that reduces the cache miss penalty by anticipating the cache behavior based on previous execution. Our observations indicate that instruction cache misses often repeat in clusters under certain conditions prevalent in real time embedded networking systems. By identifying the start of a cluster miss sequence and preparing an instruction buffer for the upcoming cache misses, the miss penalty can be reduced if a miss does occur. A sample industrial networking example is used to illustrate the effectiveness of this technique compared with other prefetch methods. In modern embedded systems, optimizing the cache performance can be crucial to achieving a marketable design. Embedded systems designers must live within tight constraints to maximize performance in a low cost system. Embedded systems usually run dedicated applications, but these dedicated applications may not be well suited to traditional cache architectures, which are designed for general purpose applications.. Fortunately, embedded systems can be tuned to use smaller, more efficient cache architectures better suited for dedicated applications.. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. This paper focuses on one specific dedicated application ─ realtime embedded networking. This application is often data driven and reactive in nature, and must operate under real time processing constraints where the software behavior can change radically, depending on the system load and packet mix behavior at a particular instance in time [14, 20] .. Features such as interrupt processing, cryptographic algorithms, and context switching make traditional cache architectures less effective for embedded networking applications.. To gain the time-to-market advantage of IP reuse, embedded systems are often implemented using synthesizable CPU cores with internal caches of modest size. These commercially available cores often form the central component in an ASIC design which is combined with custom logic to comprise an embedded system [15, 21] .. Due to the small cache size, high miss rates can be quite common in such systems, meaning performance can be improved substantially if the miss penalty can be reduced.. Embedded systems are often implemented using shared internal busses to transport data between the CPU and ASIC modules [4, 22] . In a typical embedded system, a single external memory is also available which must be shared between the CPU and various ASIC modules that can perform DMA access. Unfortunately, this bus sharing adds to the overall cache miss penalty since the refill time for cache misses is adversely affected by the arbitration delay for the memory.. These unique requirements of embedded systems, particularly for networking applications, demand an efficient, low-cost cache support architecture. The technique presented here uses a Cluster Miss Prediction Buffer (CMPB) placed on the shared bus between the CPU core and the external memory (Figure 1 ), which attempts to reduce the cache fill penalty by predicting a cluster of instruction misses based on previous cache miss behavior. The CMPB also reduces the overall system bus traffic since misses are serviced by the buffer, not the external memory. In the context of embedded networking applications, we have described a new method of instruction prefetching that reduces the cache miss penalty. We have shown that instruction cache misses often occur in clusters under certain conditions prevalent in real time embedded networking systems, and that by identifying the start of a cluster miss sequence and filling an instruction buffer for the upcoming cache misses, the miss penalty can be reduced if a miss does occur. Finally, we demonstrated this method using a sample industrial networking example to compare its effectiveness compared to the POM prefetch method.. A key observation is that our CMPB method is highly efficient. For the sample application, the selected clusters resulted in no inaccurate prefetches and hence did not contribute to wasted bus bandwidth. CMPB was highly selective in terms of what it chose to prefetch as compared to the POM techniques. These results show that there is a tradeoff between aggressive prefetching vs. selective prefetching with regard to performance, where CMPB gives decent performance boost with much less prefetch activity.. This observation leads us to speculate that the CMPB technique could be used to compliment POM technique, with clusters selected only in cases where POM would fail to predict the cache miss sequence (e.g. code that does not execute sequentially). For example, if a cache miss is not part of an identified CMPB cluster, it could degenerate to POM to service the Active Miss FIFO (Figure 4 ) rather than CMPB. Current experimentation using fully randomized packet mixes has shown CMPB combined with POM to be a promising technique with regards to dynamic behavior. Additional experimentation will also attempt to quantity the improvement in WCET when using CMPB method. Embedded system designers often employ a variety of different techniques to improve CPU instruction cache performance. Many of these are quite effective and often are focused on solving the problem with regard to one or more of the following factors: low power, lost cost (silicon area), or high performance [2] .. A common theme of many of these techniques is the inclusion of a prefetch buffer external to the CPU core and L1 cache. Prefetching attempts to reduce memory latency associated with a cache miss by predicting upcoming cache line accesses. If an upcoming cache line is not currently cached in the CPU, a good prefetch method can load the cache line into the cache now and prevent a miss later. A related technique also fetches the upcoming cache line from main memory but stores it into the prefetch buffer instead, commonly known as a partial miss.. A classical concept often used to service the prefetch buffer uses streaming buffers that prefetch the next sequential line on a miss [9, 11] . These techniques attempt to exploit the temporal and spatial locality of instruction accesses, since the next cache access often follows sequential execution order.. Unfortunately, most prefetch techniques are not as effective for embedded networking. First, most of these techniques suffer in the presence of program branches. While there are some prefetch techniques that handle branches using dynamic prediction [3, 17] , those methods can not be used effectively in embedded systems employing CPU cores since they require access to information deep in the processor core that may not be exposed outside the core (e.g., branch prediction tables, branch history tables, or pipeline state). Moreover, the unpredictable instruction flow behavior of embedded networking leads to many context switches and interrupt vectoring, again rendering prefetching less effective.. The real-time nature of embedded networking also makes prefetching difficult. As a result, real time systems use techniques such as cache line locking, column caching, software prefetching, and scratch pad memory to improve predictability for time critical portions of code [1, 16] . Methods used to quantify, improve and bound the worst case execution time (WCET) of real time systems are described in [6, 10] . These methods exploit the fixed nature of the embedded applications to tune the execution flow to the embedded memory organization, allowing improved predictability of performance in a multi-tasking environment.

O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Automated application of fault tolerance mechanisms in a component-based system Due to the reduction of structure sizes in modern embedded systems, tolerating soft errors presenting itself as bit flips becomes a mandatory task even for moderate critical applications. Accordingly, software-based fault tolerance mechanisms recently gained in popularity and a multitude of approaches that differ in the number and frequency of tolerated errors as well as their associated overhead have been proposed. As a consequence, an application-and environment-tailored selection of mechanisms is required to balance protection and costs.Accounting the diverse solution space, we propose to make software-based fault tolerance a matter of configuration that should be transparent to the applications. While this would be cumbersome when using an unsafe programming language, we show that in the context of KESO, a JVM for deeply embedded systems, this can be achieved by utilizing the Java type system and static code analysis. As an initial technique we decided to add redundant execution to KESO, which enables us to selectively and transparently replicate an application. This essentially builds a first step to a JVM, which offers reliable execution of components as demanded by the system configuration. The ability to tolerate soft errors has become an important aspect in safety-critical embedded systems. Such errors occur randomly and have -in contrast to hard errors -only a temporary effect on logical circuits or memory. Soft errors are a result of hardware failures that are becoming more likely to happen as a consequence of shrinking structure sizes. Also, they can be caused by extreme environmental conditions such as radiation [12] or current supply problems.. The developers of safety-critical systems are aware of these dependability issues as can be seen by the release of standards such as IEC 61508 or ISO 26262, which is widely spread in the automotive sector. These standards outline hardwarebased redundancy and the employment of specialized errorcorrecting hardware components -such as ECC for memory devices or hardware watchdogs to recognize bogus behavior of components -as a possible solution.. However, this approach is often not feasible due to an immense cost pressure in this field, which manifests in the consolidation of the count of microcontrollers and the integration of mixed-criticality applications on a single electronic control unit (ECU). Besides the cost factor, hardware redundancy is also often impractical due to physical size, weight, and power constraints, which are eventually an essential requirement in unmanned flight, for example.. Software-based fault tolerance techniques such as replicated execution of code in space and time or monitoring software components pose a cheaper alternative to hardwarebased techniques to increase system dependability and a combination of both approaches might also be sensible. Softwarebased approaches have already been employed in a wide range of safety-critical embedded systems, where the dominating programming language is C or Assembler. Due to the lack of type safety of C, application software has to be especially prepared for the use of many fault tolerance mechanismsthat are profoundly application-specific -such as control flow monitoring [4] or replicated execution. In case of replication, data distribution, voting and recovery mechanisms have to be manually inserted into the code base, since an identification of the data to be compared as well as the state of a replica is not possible in unsafe languages without additional support. By means of static analysis of the application -which is implemented in a type-safe language -and the support for software component isolation, fault tolerance mechanisms can automatically be applied as demanded by the safety description without the need to change the application. This is particularly important, since reliability is a non-functional property that affects the entire software system and it is still an open question, which fault tolerance mechanisms are most suitable to mask certain hardware failures. Therefore, the kind of fault tolerance techniques and the location where to apply them should be a matter of configuration that is based on the safety requirements. These requirements are usually derived from test executions, where hardware faults to be tolerated are simulated. Also, the fault tolerance mechanisms themselves have to be tailored to the actual hardware platform and the application to be deployed by static code analysis to balance protection and costs.. We address these challenges by extending the KESO Java Virtual Machine (JVM) [27] -an ahead-of-time Java-to-C compiler for static software modules -to automatically weave fault tolerance measures into the C code as described in the system configuration. So far, KESO supports transparent N-redundant execution of application modules. This is a first step towards a JVM, that offers configurable fault tolerance for application software and is itself reliable.. The approach of providing redundant execution by KESO gains several benefits over a manual implementation in safetycritical control software in unsafe languages:. • Software-based spatial isolation in combination with hardware-based memory protection of replicas inhibits transient errors to affect code outside the replicated component.. • Fault tolerance at option: Configurable number of replicas, since this is depends on the number of transient faults that need to be tolerated, which is in turn related to the safety requirements of the application. Generally, the choice for a particular fault tolerance technique as well as how and the location where this technique is applied should be a configurable property. Initialization, activation (i.e., including data replication) and synchronization of replicas are performed by the KESO runtime system.. • Automated fault tolerant replication: Generation and invocation of application-specific voting functions as well as state recovery of replicas in case of the presence of transient errors due to type-safety of Java and aheadof-time analysis of the application.. • Enforcement of replica determinism: Static analysis, to determine if code to be replicated satisfies certain criteria such as use of deterministic functions.. In the following sections, the system and fault model of a safety-critical control application, which shall be safeguarded by replicated execution, is described. Afterwards, we present the KESO JVM and how replicated execution can be integrated into this system including a discussion on possible solutions. In addition, the partially implemented approach is evaluated on a safety-critical control algorithm. This preliminary evaluation is followed by a discussion of related work in this field. To conclude this paper, we present our contributions and future work. In this paper, we presented how the N-redundant execution technique can be integrated into a Java ahead-of-time compiler. Currently, a full support of the proposed replicated execution and variants -such as the pair-and-spare technique -is in progress. Fault tolerance methods impose more overhead in space and time to improve the system reliability by their nature. However, a KESO applicationwhich can also be a device driver, for example -can select these methods according to the safety requirements and only pays for the fault tolerance features that it actually needs. Functional safety is a non-functional property that should be transparent to the applications and the required level of safety should be a matter of configuration. Static analysis on software components that are implemented in any type-safe language and the support for spatial isolation facilitate to automatically apply configurable fault tolerance techniques that are tailored to the application's requirements. As future multicore embedded systems will become less reliable due to shrinking structure sizes, the available parallelism of these platforms can also be used to compensate the overhead imposed by the application of fault tolerance techniques. Our next steps will be to integrate more fault tolerance measures into KESO to be applied automatically as demanded by the safety configuration and to evaluate them with respect to performance and footprint on the characteristics of static embedded applications. We also have to test the effectiveness of our approach with respect to tolerated and undetected errors by means of fault injection. This comprises an evaluation on several voting techniques including the consideration of application-specific SPOFs, automated control flow monitoring -as control flow errors make up 33%-77% of all runtime errors depending on the application [5] -and the use of checksums. We will also analyze the use of a more finegrained level of redundancy, that is, a software component does not have to be replicated as a whole, but only certain parts of it. This helps to determine the best possible rate between memory and execution time costs with its effects on dependability in mind. Transient errors can affect user applications as well as system services, that is, also the OS or the KESO runtime system can be compromised by bit flips. For this, fault tolerance techniques have also to be applied to the virtual machine itself, which comprises the use of a fault tolerant GC, for example. As we want to rely on existing software standards and since there is currently no implementation of a type-safe AUTOSAR OS, which can benefit from KESO directly, the OS itself has to take care of soft errors. Traditionally, handling of soft errors is addressed by safety critical applications that are executed in harsh environments such as found in the aerospace domain. Here, the high costs of hardware-centric approaches like double or triple hardware redundancy [10, 25, 30, 31] can be tolerated.. To reduce costs and make soft error tolerance available to a wider range of application scenarios, software-based fault handling methods for control flow [15, 16, 14, 17, 29, 22, 6] or memory protection [23] were developed over the recent years. Lately, Chang et al. [21] proposed SWIFT-R, an approach that triplicates machine instructions and uses a majority voter in combination with encoding techniques.. Unlike these approaches that operate at the level of machine instructions we propose tolerating soft errors at the level of components. This avoids dependence on a custom compiler for the target platform and enables to selectively protect critical parts of applications.. There are several approaches that apply replication at a higher level of abstraction. In case of Shye et al. an approach is presented that replicates processes in a standard operating system [24] . While transparent application of software-fault tolerance is possible this way it is rather coarse grained and only a single technique is used.. More in line with our vision of a soft error hardening of applications is the work of Afonso et al. [1] , that selectively enhances an embedded real-time system with fault tolerance mechanisms by using aspect-oriented programming (AOP). In contrary to Afonso who addresses a typical C++ application on an embedded system and therefore requires extensive knowledge about the application when applying fault tolerance mechanisms our approach strongly benefits from the modular design of KESO applications.. At the level of Java virtual machines the work of Napper et. al [13] and Friedman et. al [7] focus on the replication of a virtual machine as a whole. Their work is dedicated towards tolerating fail stop faults in a distributed setting but does not address soft errors on a single system. Control software can be split into parts that are controlled by replication and those which are not. This is the same approach as preparing software for software-based memory protection without any replication. The parts of the application to be executed in non-replicated mode as well as those parts to be safeguarded are configured to reside in different domains in the KESO system configuration. The domains Srv R1, Srv R2 and Srv R3 export the service process, which shall be processed redundantly, whereas the activating client application in domain A statically imports this service. . The KESO runtime system automatically creates multiple instances of the service object, one in each of the replica domains. Each replicated domain is initialized by KESO and has its own static fields and heap.. For service replication, we leverage the portal mechanism of KESO, which is depicted in Figure 3 . A service is defined by a unique name, which is used by the client domain to refer to that service, a Java interface, and the implementation of that interface. If the software already uses software-based isolation to protect the important part from other less critical code sections, the application does not need to be touched at all. In the other case, the regular method call that usually starts the control procedure has to be replaced by a portal call. If neither replication nor isolation is required, the portal call is optimized to a regular call by jino, thus erasing any overhead.

O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Histogram Equalization Utilizing Window-Based Smoothed CDF Estimation for Feature Compensation In this letter, we propose a new histogram equalization method to compensate for acoustic mismatches mainly caused by corruption of additive noise and channel distortion in speech recognition. The proposed method employs an improved test cumulative distribution function (CDF) by more accurately smoothing the conventional order statisticsbased test CDF with the use of window functions for robust feature compensation. Experiments on the AURORA 2 framework confirmed that the proposed method is effective in compensating speech recognition features by reducing the averaged relative error by 13.12% over the order statisticsbased conventional histogram equalization method and by 58.02% over the mel-cepstral-based features for the three test sets. Speech recognizers trained on clean speech data usually show dramatic degradation of recognition accuracy when they are operated in real-world noisy environments. The main cause of performance degradation is acoustic mismatches between clean training and noisy test environments [1] , [2] . The histogram equalization (HEQ) technique is known to be one of the most efficient methods with very comparable effectiveness in compensating for the acoustic mismatch in the feature space of speech recognition [3] - [5] . In the histogram equalization approach, reliable and accurate estimation of cumulative distribution functions (CDFs) is a critical issue for guaranteeing its effectiveness in the feature compensation [5] . We can use a sufficient amount of training data in the reference (or training) CDF estimation. Therefore, the reference CDF can be obtained highly accurately by using the training data. However, the test CDF is apt to be estimated with a limited amount of test data because the basic input unit in most of the current speech recognizers is usually a short utterance or word. As the amount of test data is smaller, the accuracy of the estimated probability distributions tends to be deteriorated further. Therefore, the classical histogram method is not suitable for the test CDF estimation in short utterance-based test environments. As an improved test CDF estimation method, the order statistics-based approach is widely employed in the histogram equalization techniques [3] . However, from the viewpoint of kernel density estimation [6] it is regarded that the order statistics-based method uses a kernel function with a very small bandwidth which covers only a single sample data. As a result, the resulting CDF by the order statistics-based method tends to be extremely under-smoothed. Therefore, by utilizing a reasonable kernel function with a properly chosen bandwidth, we can obtain a more accurate test CDF. Finally, it is surely expected that histogram equalization based on the enhanced test CDF provides better effectiveness in the feature compensation.. In this letter, we propose a new histogram equalization technique which employs a window-based smoothed test CDF estimation method for feature compensation in the noise robust speech recognition. Our method utilizes a window-type smoothing function as the kernel function in the test CDF estimation instead of just using the rank information of each sample data in the conventional order statistics-based method and produces a more accurate test CDF. The experimental results showed that the proposed approach is more effective than the conventional order statistics-based histogram equalization (OS-HEQ) technique in compensating the acoustic features in noisy environments. For more effective feature compensation in noisy speech recognition environments, the histogram equalization technique requires more accurate test CDF estimation. The order statistics-based CDF estimation is the most frequently employed approach in the conventional histogram equalization techniques, but it still has some room for further improvement in its accuracy. The window-based smoothed CDF estimation method can obtain more accurate test CDF by smoothing the order statistics-based CDF with the use of window functions. The proposed histogram equalization technique then utilizes the improved test CDF estimated by the window-based smoothed estimation method. Experimental results showed the effectiveness of the proposed histogram equalization technique for feature compensation in robust speech recognition.

B I O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O B I O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O B I O B I I