Designing application-specific networks on chips with floorplan information With increasing communication demands of processor and memory cores in Systems on Chips (SoCs), scalable Networks on Chips (NoCs) are needed to interconnect the cores. For the use of NoCs to be feasible in today's industrial designs, a custom-tailored, application-specific NoC that satisfies the design objectives and constraints of the targeted application domain is required. In this work, we present a design methodology that automates the synthesis of such application-specific NoC architectures. We present a floorplan aware design method that considers the wiring complexity of the NoC during the topology synthesis process. This leads to detecting timing violations on the NoC links early in the design cycle and to have accurate power estimations of the interconnect. We incorporate mechanisms to prevent deadlocks during routing, which is critical for proper operation of NoCs. We integrate the NoC synthesis method with an existing design flow, automating NoC synthesis, generation, simulation and physical design processes. We also present ways to ensure design convergence across the levels. Experiments on several SoC benchmarks are presented, which show that the synthesized topologies provide a large reduction in network power consumption (2.78× on average) and improvement in performance (1.59× on average) over the best mesh and mesh-based custom topologies. An actual layout of a multimedia SoC with the NoC designed using our methodology is presented, which shows that the designed NoC supports the required frequency of operation (close to 900 MHz) without any timing violations. We could design the NoC from input specifications to layout in 4 hours, a process that usually takes several weeks. To have a power and latency efficient design, the communication architecture should closely match the application traffic characteristics, satisfying the different design constraints. Synthesizing such Network on Chip (NoC) architecture is non-trivial, given the large design space that needs to be explored. In this work, we have presented a methodology that automates the process, generating efficient NoCs that satisfy the design constraints of the application. To have fewer design re-spins and faster time-to-market, we consider fast and accurate floorplan information early in the design cycle. This leads to detecting timing violations on the NoC links during the NoC synthesis phase, thereby leading to timing closure with quicker convergence between the high level design and the physical design phases. We use accurate switch and link power models that are based on layouts of the components and accurate link power estimates based on the wire-lengths obtained from floorplanning. We also integrate deadlock free routing methods in the NoC synthesis process, which is critical for proper NoC operation. Experiments on several SoC benchmarks show that the synthesized topologies are much better (an average of 2.78× power reduction, 1.59× hop-count reduction) than the best mesh topology and meshbased custom topologies for our case studies. A large body of research works exists in synthesizing and generating bus-based systems [9] - [14] . A floorplan-aware point-to-point link design and bus design methodologies are presented in [15] and [14] . While some of the design issues in the NoCs are similar to bus based systems (such as link-width sizing), a large number of issues such as finding the number of required switches, sizing the switches, finding routes for packets, etc. are new in NoCs.. Methods to collect and analyze traffic information that can be fed as input to the bus and NoC design processes have been presented in [12] and [13] . Mappings of cores onto standard NoC topologies have been explored in [16] - [19] . In [17] , [19] a floorplanner is used during the mapping process to get area and wire-length estimates. Unlike the method presented here, these works only select topologies from a library of standard topologies. In [18] , a unified approach to mapping, routing and resource reservation has been presented. However, the work does not explore topology design process. The NoC design process for supporting multiple applications has been presented in [20] . This research complements our work and its methods can be applied here to support multiple applications as well.. Important research in macro-networks has considered the topology generation problem [21] . As the traffic patterns on these networks are difficult to predict, most approaches are tree-based (like spanning or Steiner trees) and only ensure connectivity with node degree constraints [21] . Hence, these techniques cannot be directly extended to address the NoC synthesis problem. Applicationspecific custom topology design has been explored earlier in [22] - [25] . The works from [22] , [23] do not consider the floorplanning information during the topology design process. In [24] , a physical planner is used during topology design to reduce power consumption on wires. However, the work does not consider the area and power consumption of switches in the design. Also, the number and size of network partitions are manually fed. In [25] , a slicing tree based floorplanner is used during the topology design process. This work assumes that the switches are located at the corners of the cores and it does not consider the network components (switches, network interfaces) during the floorplanning process. Also, deadlock free routing, which is critical for custom NoC designs is not supported in the work. Moreover, a complete design space exploration, from architectural parameter setting to simulation is not presented.. Several works exist on automatically generating the Register Transfer Level (RTL) code of a designed topology for simulation and synthesis [26] - [28] . These works again complement ours, as the input to them is a designed topology. Building area, power models for on-chip networks has been addressed in [29] - [32] .

O O B I I O B O O O B O O B O O O O O O O O O B I I O O O O B O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B B O O O B O O O O O O O O B O B B O O O O B O O O O O B B B O O O O B O O O O B O O O B O O O O O B O O O B O O O O O O O B O O O O O O O O O O O O B B O O O B B O O B B B O B I O O O O O O O B B O O O O O O O O O O O O O O O O O O O B O B B I O O O O O O B O O O O O O B O O B O O O B O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O B O O O B O O B O O O O B O O O O O O O O B O O O B O B B O O O O O O O O O O B O O O B O O O B O O O O O B O O O O O O O O O O O O O B O O O B O O O O O O B O O O O O O O B O O O O O O O O O B O O O O B O O O O O B O O O O B O O O B O O O O B O O O B O O O O B O O B I O O B O O O O B O O O O O O O O O O O O B O O O O O O O O O O O B O B O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B B B O O B B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B B B O O O O O O O O O O O O O O O O O O O O O O O B B B O O O O O O O O O B B O O B I O O O O O O O O O O O B I O O O O B O O O O O O B O O O O O O O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O O O O B O O B O O O O B O B O B O O O O B O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O B O B O B O O O O O O O O O O O B O O O O O B O O B O O O O O O O O O O O O


A kinematic approach to Kokotsakis meshes A Kokotsakis mesh is a polyhedral structure consisting of an n-sided central polygon P 0 surrounded by a belt of polygons in the following way: Each side a i of P 0 is shared by an adjacent polygon P i , and the relative motion between cyclically consecutive neighbor polygons is a spherical coupler motion. Hence, each vertex of P 0 is the meeting point of four faces. In the case n = 3 the mesh is part of an octahedron. These structures with rigid faces and variable dihedral angles were first studied in the thirties of the last century. However, in the last years there was a renaissance: The question under which conditions such meshes are infinitesimally or continuously flexible gained high actuality in discrete differential geometry. The goal of this paper is to revisit the wellknown continuously flexible examples (Bricard, Graf, Sauer, Kokotsakis) from the kinematic point of view and to extend their list by a new family. The definition of a Kokotsakis mesh (German: Neunflach) dates back to A.I. Kokotsakis (1932) . While for arbitrary n the classification of continously flexible Kokotsakis meshes is an open problem (compare Bobenko et al., 2008, p. 76) , for the case n = 3 we know since R. Bricard (1897) that there are three flexible types (see also Stachel, 1987) .. In discrete differential geometry there is an interest in polyhedral structures composed of quadrilaterals, i.e., in quadrilateral surfaces. When all quadrilaterals are planar, the edges form a discrete conjugate net (see, e.g., Pottmann et al., 2007, Fig. 14e) . When each quadrilateral is seen as a rigid body and only the dihedral angles can vary, the question arises under which conditions such structures are flexible. In Bobenko et al. (2008) , p. 75, the following theorem can be found: A discrete conjugate net is continuously flexible if and only if all its 3 × 3 complexes, i.e., all included Kokotsakis meshes, are continuously flexible. This reveals the recent interest in these examples of overconstrained structures.

O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Improved Error Exponent for Time-Invariant and Periodically Time-Variant Convolutional Codes Abstract-An improved upper bound on the error probability (first error event) of time-invariant convolutional codes, and the resulting error exponent, is derived in this paper. The improved error bound depends on both the delay of the code and its width (the number of symbols that enter the delay line in parallel) . Determining the error exponent of time-invariant convolutional codes is an open problem. While the previously known bounds on the error probability of time-invariant codes led to the block-coding exponent, we obtain a better error exponent (strictly better for 1).In the limit our error exponent equals the Yudkin-Viterbi exponent derived for time-variant convolutional codes. These results are also used to derive an improved error exponent for periodically time-variant codes.

O B I O O O O O B I O O O O O O B I O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O B I O B I O O O O O O O O O B I O O O O O O O B I O O B I O O O B I O O O O O O O O O B I O B I I


An improved DFA for fast regular expression matching Modern network devices need to perform deep packet inspection at high speed for security and application-specific services. Finite Automata (FAs) are used to implement regular expressions matching, but they require a large amount of memory. Many recent works have proposed improvements to address this issue. This paper presents a new representation for deterministic finite automata (orthogonal to previous solutions), called Delta Finite Automata (δFA), which considerably reduces states and transitions and requires a transition per character only, thus allowing fast matching. Moreover, a new state encoding scheme is proposed and the comprehensive algorithm is tested for use in the packet classification area. Many important services in current networks are based on payload inspection, in addition to headers processing. Intrusion Detection/Prevention Systems as well as traffic monitoring and layer-7 filtering require an accurate analysis of packet content in search of matching with a predefined data set of patterns. Such patterns characterize specific classes of applications, viruses or protocol definitions, and are continuously updated. Traditionally, the data sets were constituted of a number of signatures to be searched with string matching algorithms, but nowadays regular expressions are used, due to their increased expressiveness and ability to describe a wide variety of payload signatures [22] . They are adopted by well known tools, such as Snort [23] and Bro [3] , and in firewalls and devices by different vendors such as Cisco [26] .. Typically, finite automata are employed to implement regular expression matching. Nondeterministic FAs (NFAs) are representations which require more state transitions per character, thus having a time complexity for lookup of O(m), where m is the number of states in the NFA; on the other hand, they are very space-efficient structures. Instead, Deterministic FAs (DFAs) require only one state traversal per character, but for the current regular expression sets they need an excessive amount of memory. For these reasons, such solutions do not seem to be proper for implementation in real deep packet inspection devices, which require to perform on line packet processing at high speeds. Therefore, many works have been recently presented with the goal of memory reduction for DFAs, by exploiting the intrinsic redundancy in regular expression sets [14, 13, 5, 21] .. This paper focuses in memory savings for DFAs, by introducing a novel compact representation scheme (named δFA) which is based on the observation that, since most adjacent states share several common transitions, it is possible to delete most of them by taking into account the different ones only. The δ in δFA just emphasizes that it focuses on the differences between adjacent states. Reducing the redundancy of transitions appears to be very appealing, since the recent general trend in the proposals for compact and fast DFAs construction (see sec.2) suggests that the information should be moved towards edges rather than states. Our idea comes from D 2 FA [14] , which introduces default transitions (and a "path delay") for this purpose.. Unlike the other proposed algorithms, this scheme examines one state per character only, thus reducing the number of memory accesses and speeding up the overall lookup process. Moreover, it is ortoghonal to several previous algorithms (even the most recent XFAs [21, 20] and H-cFA [13] ), thus allowing for higher compression rates. Finally, a new encoding scheme for states is proposed (which we will refer to as Char-State compression), which exploits the association of many states with a few input characters. Such a compression scheme can be efficiently integrated into the δFA algorithm, allowing a further memory reduction with a negligible increase in the state lookup time.. We also test the integration of δFA and Char-State compression in the packet classification area, by representing the classification rule set through regular expressions.. In summary, the main contributions of this paper are:. • a novel compact representation of DFA states (δFA) which allows for iterative reduction of the number of states and for faster string matching;. • a new state encoding scheme (Char-State compression) based on input characters;. • the application of both schemes to classification algorithms to increase search speed.. The remainder of the paper is organized as follows. In section 2 related works about pattern matching and DFAs are discussed. Sec.3 describes our algorithm, by starting from a motivating example and sec.4 proves the integration of our scheme with the previous ones. Then in sec.5 the encoding scheme for states is illustrated and in the subsequent section the integration with δFA is shown. Finally, sec.8 presents the experimental results, while sec.9 proves the applicability of δFA to packet classification. In this paper, we have presented a new compressed representation for deterministic finite automata, called Delta Finite Automata. The algorithm considerably reduces the number of states and transitions and it is based on the observation that most adjacent states share several common transitions, so it is convenient to store only the differences between them. Furthermore, it is orthogonal to previous solutions, this way allowing for higher compression rates. Another fundamental feature of the δFA is that it requires only a state transition per character (keeping the characteristic of standard DFAs), thus allowing a fast string matching.. A new encoding scheme for states has been also proposed (that we refer to as Char State), which exploits the association of many states with a few input chars. Such a compression scheme can be efficiently integrated into the δFA algorithm, allowing a further memory reduction with a negligible increase in the state lookup time.. Finally, the integration of both schemes has also been proposed for application in the field of packet classification, by representing the classification rule set through regular expressions. The experimental runs have shown good results in terms of lookup speed as well as the issue of excessive memory consumption, which we plan to address by exploiting the recent techniques presented in [13] and [21, 20] . Deep packet inspection consists of processing the entire packet payload and identifying a set of predefined patterns. Many algorithms of standard pattern matching have been proposed [1, 9, 27] , and also several improvements to them. In [24] the authors apply two techniques to Aho-Corasick algorithm to reduce its memory consumption. In details, by borrowing an idea from Eatherton's Tree Bitmap [10] , they use a bitmap to compress the space near the root of the state machine, where the nodes are very dense, while path compressed nodes and failure pointers are exploited for the remaining space, where the nodes become long sequential strings with only one next state each.. Nowadays, state-of-the-art systems replace string sets with regular expressions, due to their superior expressive power and flexibility, as first shown in [22] . Typically, regular expressions are searched through DFAs, which have appealing features, such as one transition for each character, which means a fixed number of memory accesses. However, it has been proved that DFAs corresponding to a large set of regular expressions can blow up in space, and many recent works have been presented with the aim of reducing their memory footprint. In [28] the authors develop a grouping scheme that can strategically compile a set of regular expressions into several DFAs evaluated by different engines, resulting in a space decrease, while the required memory bandwidth linearly increases with the number of active engines.. In [14] , Kumar et al. introduce the Delayed Input DFA (D 2 FA), a new representation which reduces space requirements, by retrieving an idea illustrated in [2] . Since many states have similar sets of outgoing transitions, redundant transitions can be replaced with a single default one, this way obtaining a reduction of more than 95%. The drawback of this approach is the traversal of multiple states when processing a single input character, which entails a memory bandwidth increase to evaluate regular expressions.. To address this issue, Becchi and Crowley [6] introduce an improved yet simplified algorithm (we will call it BEC-CRO) which results in at most 2N state traversals when processing a string of length N . This work is based on the observation that all regular expression evaluations begin at a single starting state, and the vast majority of transitions among states lead back either to the starting state or its near neighbors. From this consideration and by leveraging, during automaton construction, the concept of state distance from the starting state, the algorithm achieves comparable levels of compression with respect to D 2 FA, with lower provable bounds on memory bandwidth and greater simplicity.. Also, the work presented in [4] focuses on the memory problem of DFAs, by proposing a technique that allows nonequivalent states to be merged, thanks to a scheme where the transitions in the DFA are labeled. In particular, the authors merge states with common destinations regardless of the characters which lead those transitions (unlike D 2 FA), creating opportunities for more merging and thus achieving higher memory reduction. Moreover the authors regain the idea of bitmaps for compression purposes.. Run-Length-Encoding is used in [7] to compress the transition table of DFAs. The authors show how to increase the characters processed per state traversal and present heuristics to reduce the number of memory accesses. Their work is specifically focused on an FPGA implementation.. The work in [5] is based on the usual observation that DFAs are infeasible with large sets of regular expressions (especially for those which present wildcards) and that, as an alternative, NFAs alleviate the memory storage problem but lead to a potentially large memory bandwidth requirement. The reason is that multiple NFA states can be active in parallel and each input character can trigger multiple transitions. Therefore the authors propose a hybrid DFA-NFA solution bringing together the strengths of both automata: when constructing the automaton, any nodes that would contribute to state explosion retain an NFA encoding, while the others are transformed into DFA nodes. As shown by the experimental evaluation, the data structure presents a size nearly that of an NFA, but with the predictable and small memory bandwidth requirements of a DFA.. Kumar et al. [15] also showed how to increase the speed of D 2 FAs by storing more information on the edges. This appears to be a general trend in the literature even if it has been proposed in different ways: in [15] transitions carry data on the next reachable nodes, in [4] edges have different labels, and even in [13] and [21, 20] transitions are no more simple pointers but a sort of "instructions".. In a further comprehensive work [13] , Kumar et al. analyze three main limitations of the traditional DFAs. First, DFAs do not take advantage of the fact that normal data streams rarely match more than a few initial symbols of any signature; the authors propose to split signatures such that only one portion needs to remain active, while the remaining portions can be "put to sleep" (in an external memory) under normal conditions. Second, the DFAs are extremely inefficient in following multiple partially matching signatures and this yields the so-called state blow-up: a new improved Finite State Machine is proposed by the authors in order to solve this problem. The idea is to construct a machine which remembers more information, such as encountering a closure, by storing them in a small and fast cache which represents a sort of history buffer. This class of machines is called History-based Finite Automaton (H-FA) and shows a space reduction close to 95%. Third, DFAs are incapable of keeping track of the occurrencies of certain sub-expressions, thus resulting in a blow-up in the number of state: the authors introduce some extensions to address this issue in the History-based counting Finite Automata (H-cFA).. The idea of adding some information to keep the transition history and, consequently, reduced the number of states, has been retrieved also in [21, 20] , where another scheme, named extended FA (XFA), is proposed. In more details, XFA augments traditional finite automata with a finite scratch memory used to remember various types of information relevant to the progress of signature matching (e.g., counters of characters and other instructions attached to edges and states). The experimental tests performed with a large class of NIDS signatures showed time complexity similar to DFAs and space complexity similar to or better than NFAs. The implementation of δFA and C-S should be adapted to the particular architecture of the hardware platform. However, some general guidelines for an optimal deployment can be outlined. In the following we will make some general assumptions on the system architecture; such assumptions are satisfied by many network processing devices (e.g. the Intel IXP Network Processors [12] ). In particular, we assume our system to be composed by:. • a standard 32 bit processor provided with a fairly small local memory (let us suppose a few KBs); we consider the access time to such a memory block to be of the same order of the execution time of an assembly level instruction (less than ten clock cycles);. • an on-chip fast access memory block (which we will refer to as scratchpad) with higher storage capacity (in the order of 100 KB) and with an access time of a few dozens of clock cycles;. • an off-chip large memory bank (which we will refer to as external memory) with a storage capacity of dozens of MBs and with an access time in the order of hundreds of clock cycles.. We consider both δFA and Char-State compression algorithms. As for the former, two main kinds of data structures are needed: a unique local transition set and a set of data structures representing each state (kept in the external memory). The local transition set is an array of 256 pointers (one per character) which refer to the external memory location of the data structure associated with the next state for that input char; since, as reported in table 3.b , the memory occupation of a δFA is generally smaller than 1 MB, it is possible to use a 20 bit-long offset with respect to a given memory address instead of an actual pointer, thus achieving a consistent compression. A δFA state is, on the contrary, stored as a variable-length structure. In its most general form, it is composed by a 256 bit-long bitmap (specifying which valid transition are already stored in the local transition set and which ones are instead stored within the state) and a list of the pointers for the specified transitions, which, again, can be considered as 20 bit offset values.. If the number of specified transitions within a state is small enough, the use of a fixed size bitmap is not optimal: in these cases, it is possible to use a more compact structure, composed by a plain list of character-pointer couples. Note that this solution allows for memory saving when less than 32 transitions have to be updated in the local table.. Since in a state data structure a pointer is associated with a unique character, in order to integrate Char-State compression in this scheme it is sufficient to substitute each absolute pointer with a relative-id. The only additional structure consists of a character-length correspondence list, where the length of the relative-ids associated with each character is stored; such an information is necessary to parse the pointer lists in the node and in the local transition set. However, since the maximum length for the identifiers is generally lower than 16 bits (as it is evident from figure 4), 4 bits for each character are sufficient. The memory footprint of the character-length table is well compensated by the corresponding compression of the local transition set, composed by short relative identifiers (our experimental results show a compression of more than 50%). Furthermore, if a double indirection scheme for the translation of relative-ids is adopted, a table indicating the number of unique identifiers for each character (the threshold value we mentioned in section 5.1) will be necessary, in order to parse the indirection table. This last table (that will be at most as big as the compressed local transition table) can be kept in local memory, thus not affecting the performance of the algorithm.

O O B O O O O O O O O O O O B I I O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


QNoC: QoS architecture and design process for network on chip have been proposed as a solution for the problem of global interconnect in deep submicron VLSI Systems-on-Chip (SoC). Networks on Chip (NoC) can address and contain major physical issues such as synchronization, noise, error-correction and speed optimization. NoC can also improve design productivity by supporting modularity and reuse of complex cores, thus enabling a higher level of abstraction in architectural modeling of future systems [4] , [5] . However, VLSI designers must be ensured that the benefits of NoC do not compromise system performance and cost [8] , [10] . Performance concerns are associated with latency and throughput. Cost concerns are primarily chip-area and power dissipation. This paper presents a design process and a network architecture that satisfy Quality of Service (performance) requirements at a measurable VLSI cost which is favorably compared with alternative on-chip interconnection approaches.Traditionally, on-chip global communication has been addressed by shared-bus structures and ad-hoc direct interconnections. Non-scalability of these approaches was discussed in [1] , [6] , [9] . However, modern on-chip buses have evolved to multi-layered and segmented structures, supporting split transactions, burst transfers and parallel operations [12] [13] [14] . From several aspects they can be considered as networks but still, they don't provide effective spatial reuse of resources and do not utilize packet or wormhole switching associated with distributed routing and congestion/flow control. Therefore, they are inefficient and require centralized arbitration mechanisms.Advantages of spatial-reuse packet/wormhole switched networks were analyzed in comparison with buses by several authors [1] , [3] , [5] , [8] , [9] . A hybrid approach, supporting both NoC and on-chip buses has been proposed in [10] . Switched networks and techniques for their design have been developed for computer networks and for multiprocessor systems [15] [16] [17] [18] [19] [20] [21] . However, a unique set of resource constraints and design considerations exists for an on-chip environment. As described in [1] , [9] , memory and computing resources are relatively more expensive on-chip, while relatively more wires are available. The need to combine several types of service, such as "best effort" and "guaranteed throughput" was noted by [1] , [8] . In [9] it was suggested to support several access paradigms such as request-response (for compatibility with bus-based approaches) and connection-oriented services for extended functionality. A mesh network topology was proposed in [4] Unlike computer networks which are built for on-going expansion, future growth and standards compatibility, on-chip networks can be designed and customized for an a-priori known set of computing resources, given pre-characterized traffic patterns among them. These imply that various components of the network architecture including addressing fields and QoS classification can be modified between implementations. Moreover, placement of the computing resources can be made simultaneously with the design of the network. Dynamic changes of links (link upgrades or failures) are not expected on-chip. Also, highly reliable link operation can be assumed, at least in the early generations of NoCs.Based on the above considerations, given the cost sensitivity and the need to support various services and access paradigms, we suggest a QNoC architecture and a process for its design, using the following characteristics: The modules are interconnected by a network of multi-port switches connected to each other by links composed of parallel point-to-point lines. The physical layer is optimized to take care of deep-submicron issues such as delay and repeater optimization, synchronization, noise immunity etc. The network applies a mesh topology and employs wormhole packet forwarding with hop-by-hop credit-based backpressure flowcontrol (for lossless buffer operation and minimal buffer requirements). The packets are forwarded using a static shortest path, X-Y coordinates-based routing (for minimal routing table operations, deadlock avoidance and no reordering at end-points). Packets can belong to different classes of service and packets of different classes are forwarded in an interleaved manner according to the QoS definitions (packets priorities). As a typical guideline we classify system traffic into four common classes of service: Signaling (replacing inter-module control signals); Real-Time (representing delay constrained bit streams); R/W (modeling short data access) and Block-Transfer (providing for large data bursts and DMA operations). We model traffic behavior for each class and define the Quality of Service requirements of each class in terms of throughput, end-to-end delay and relative priority. Unlike other wormhole routing systems these requirements are recorded at all switches and different packet forwarding is interleaved according to the QoS rules. For example a high priority Signaling packet will preempt the transmission of a long Block-Transfer packet. Similar to [4] we employ a design process, starting from a generic topology and proceeding to a customized network. The layout of the network is customized and links bandwidth is allocated according to their relative load so that the overall utilization of links in the network is balanced. During customization unnecessary resources (links, routers and buffers) are trimmed where possible, resulting in a low cost customized layout for the specific SoC. Traffic simulations are used in the cost optimization steps to ensure that QoS is satisfied. We also introduce a simple methodology to evaluate the cost in area and power of the resulting network. The area cost is based on the total wire length and the amount of packet switch logic (buffers, tables etc.), that is estimated by the flip-flop count. The power cost is based on summation of the traffic that traverses each wire length and is received by input stages. It is easy to realize that for a given traffic metric and network topology, the power consumption will be reduced by employing shortest path routing and by the elimination of packets losses within switches. In that way our design process results in QNoC which meets predefined QoS requirements from the networking point of view and takes VLSI costs into consideration by minimizing area and power of the resulting QNoC.The rest of this paper is organized as follows: Section II presents the network architecture, Section III describes the network design process, Section IV discusses simple design examples, and Section V provides simulation results for the network examples along with observations and conclusions.Our QoS network architecture is based on a grid topology and wormhole packet routing, following [1], [6] , [7] . Wormhole routing [1], [22] reduces latency and buffer requirements in the routers. Circuit switching [7] is avoided in our architecture due to the high cost of establishing and managing circuit connections. Similarly, store-and-forward routing techniques [7] are also avoided as they may incur high buffer requirements and consequently a high penalty in silicon area of the router. The network is lossless, and links are assumed reliable so that no retransmissions are required 1 . Packets traverse the network along the shortest route, thus minimizing power dissipation and maximizing network resource utilization. The architecture has served as a platform for developing the QNoC design process and cost metrics (Section III), and has been modeled and simulated, as described in Section IV. In this section we describe the QNoC topology, service levels, link communications, router design and interface to the system modules of the chip.Networks on chip comprise routers interconnected by point-to-point links. Topology can vary depending on system needs and module sizes and placement. Fat tree [6], folded torus [1] and regular mesh [4] , [7] topologies have been proposed for NoC ( Figure 1 ). We propose an irregular mesh topology as a best match for the typically irregular planar and rectangular structure of common SoC floorplans. Each system module is connected to a router ( Figure 1c ) via a standard interface, where the bandwidth is adapted to the communication needs of the module. The bandwidth of each inter-router link is similarly adjusted to accommodate the 1 Since higher noise vulnerability is expected in future technologies, simple error correction techniques can be applied in physical, data-link or even higher levels. NoC may be capable of containing such errors and efficiently isolating them from the rest of the SoC. These techniques are beyond the scope of this paper.3 expected traffic and fulfill QoS requirements at the specific link. Link and interface bandwidth is adjustable by changing either the number of wires or the data frequency. In addition, a module may be connected to the network through more than one interface.Routing is performed over fixed shortest paths, employing a symmetric X-Y discipline whereby each packet is routed first in an "X" direction and then along the perpendicular dimension or vice versa 2 . This scheme leads to a simple, cost-effective router implementation. Network traffic is thus distributed non-uniformly over the mesh links, but each link's bandwidth is adjusted to its expected load, achieving an approximately equal level of link utilization across the chip. The principal goal of an on-chip interconnection network is to provide for all communication demands of heterogeneous modules within the chip. A QNoC should replace not only shared buses but also other types of dedicated inter-modular wires and interfaces. We identify four different types of communication requirements and define appropriate service levels (SL) to support them: Signaling, Real-Time, Read/Write and Block-Transfer. Two classes of service were previously proposed in this context in [7] : best-effort and guaranteed throughput.Signaling covers urgent messages and very short packets that are given the highest priority in the network to assure shortest latency. This service level is suitable for interrupts and control signals and alleviates the need for dedicating special, single-use wires for them.Real-Time service level guarantees bandwidth and latency to real-time applications, such as streamed audio and video processing. This service (like all other ones) is packet based (and does not employ virtual circuits); a certain maximal level of bandwidth may be allocated to each real-time link and it should not be exceeded. This is achieved either by the design of each module or by enforcement circuits in the network.Read/Write (RD/WR) service level provides bus semantics and is hence designed to support short memory and register accesses.Block-Transfer service level is used for transfers of long messages and large blocks of data, such as cache refill and DMA transfers.A priority ranking is established among the four service levels, where Signaling is given the highest priority and Block-Transfer the lowest. Below we describe a preemptive communication scheduling where data of a higher priority packet is always transmitted before that of a lower service level (a round-robin is employed within each service level). Thus, service levels are simply implemented by means of a priority mechanism. Additional service levels may be defined if desired, as long as a priority ranking is adhered to. For instance, the RD/WR service level may be split into normal and urgent RD/WR sub-levels.Packets carry routing information, command and payload. Figure 2 shows the basic packet format. The TRA (Target Routing Address) field contains the address required for routing. The command field identifies the payload, specifying the type of operation. The rest is an arbitrary length payload, including operation-specific control information such as sender identification. Routers connect to up to five links ( Figure 3 ), designed for planar interconnect to four mesh neighbors and to one chip module. The router forwards packets from input ports to output ports. Data is received in flits. Every arriving flit is first stored in an input buffer. On the first flit of a packet, the router determines to which output port that packet is destined. The router then schedules the transmission for each flit on the appropriate output port. There are separate buffers for each of the four service levels ("direct buffer mapping"). Relatively small buffers are allocated to each service level, capable of storing only a few flits (this is a tunable design parameter). The routing algorithm is invoked when the first flit of a packet is received. The algorithm uses a simple routing function. For instance, relative routing is employed for X-Y routing. Routing information per each service level per each input port is stored in the Current Routing Table (CRT; Figure 5 ), until the tail flit of the packet is received, processed and delivered. When a flit is forwarded from an input to an output port, one buffer becomes available and a buffer-credit is sent back to the previous router on separate out-of-band wires (Table2).Each output port of a router is connected to an input port of a next router via a communication link. The output port maintains the number of available flit slots per each service level in the buffer of the next input port. These numbers are stored in the Next Buffer State (NBS) table (Figure 5 ). The number is decremented upon transmitting a flit and incremented upon receiving a buffercredit from the next router. When a space is available, the output port schedules transmission of flits that are buffered at the input ports and waiting for transmission through that output port, as detailed below.We describe a simple handshake interface to each of the links. Other interfaces, such as asynchronous, are also possible. The same interface is employed whether the link connects to a chip module or to another router. The output port transmits a flit on the rising edge of the link clock (identifying the flit with a non-idle type, Table 1 ), and the input port samples a new flit on the falling edge. The clock could be replaced by a valid signal, which toggles only when a flit needs to be transmitted, alleviating the need for an "idle" flit type. Tables 1,2 We now turn to the mechanics of flit transfer inside the router. Flits are buffered at the input ports, awaiting transmission by the 6 output ports. Flit routing (namely, to which output port each flit is targeted) is resolved upon arrival of the first flit of a packet and the output port number is stored in CRT for the pending flit per each input port and per each service level. Each output port schedules transmission of the flits according to the availability of buffers in the next router, the priority (namely service level) of the pending flits, and the packet based round-robin ordering of input ports awaiting transmission of packets within the same service level. The numbers of available flit slots in the buffers at the next routers are stored in the NBS tables for each service level at each output port. Service level priorities are ranked with Signaling having the highest priority, Real-Time being second, RD/WR third and Block-Transfer ranked last. The present state of round-robin scheduling is stored in the Currently Serviced Input Port number (CSIP) table for each service level at each output port ( Figure 5 ). This number is advanced when transmission of a complete packet is finished or if there is nothing to transmit from a particular input port and service level. This scheduling discipline implies that a particular flit gets transmitted on an output port as long as there is buffer space available on the next router and there is no packet with a higher priority pending for that particular output port. Once a higher priority packet appears on one of the input ports, transmission of the current packet is preempted and the higher priority packet gets through. Transmission of the lower priority packet is resumed only after all higher priority packets are serviced. In the example of Figure 6 , a Real-Time packet is preempted by a Signaling packet. The network is designed with bounded traffic requirements in the higher service levels, to avoid starvation of RD/WR and Block-Transfer communications. CrossbarInput portsThe network interface connects modules to the network. It maps a variety of transactions onto the four structured QNoC service levels. For instance, it hides the packet switching character of the network when one module needs to access another using bus and conventional read/write semantics. This section has presented the QNoC architecture: Simple and efficient routers are interconnected with short point-to-point links and arranged as an irregular mesh. Communication is organized in four service levels; a preemptive priority scheduling is established in the router among the levels, while packet based round-robin scheduling is employed within each level. The architecture is buffer efficient due to wormhole routing and power efficient due to lossless and shortest path communication mechanism. Network resources are adjusted to fit a given traffic pattern, as described in the following section.In this section we present a design process for constructing a low cost QNoC. In traditional communication networks the topology and link capacities are given, and the routing and congestion control processes balance the a-priory unknown traffic loads. QNoC design process has more degrees of freedom as the topology of the network, network resources and protocols can be changed and tuned by the network designer for a particular SoC with particular requirements. Design effort shifts to adapting the network to given traffic flows and QoS requirements and optimizing it for low cost in terms of area and power. The block diagram in Figure 7 summarizes the QNoC design flow. First, the functional system modules are defined and assumed to be connected by an ideal interconnection infrastructure with unlimited bandwidth and programmable delay. Then, inter-module traffic is characterized. This characterization is conducted by analyzing the interconnected modules and their traffic specification. To verify the assumptions or as an alternative way for characterization, the inter-module traffic is measured and partitioned into traffic classes using a high-level multi-module operational simulation. Similarly, QoS requirements are derived for each traffic class by observing the actual performance as well as evaluating via simulation the effect of delays and throughput. The characterization and requirements derivation stage should take into account that QNoC cost will increase if safety margins are excessive. Once traffic patterns are specified, modules are placed so as to minimize the system spatial traffic density.Only after system module placement and inter-modular communication requirements are determined, the QNoC can be constructed. The QNoC architecture is finalized, and architectural parameters are set according to the number of modules, their 8 spatial placement, and the QoS service levels to be supported. The initial topology is set to a mesh grid and the required traffic is mapped onto the mesh grid according to the routing algorithm, such as X-Y routing. As parts of the grid are not fully utilized, some vertices and links can be eliminated as shown in Figure 1 .c. Once the routing algorithm is selected, communication paths between all pairs of modules can be determined and link bandwidth optimization can be performed. Average traffic load at each link can be calculated since routing is fixed and traffic patterns are known in advance. Link bandwidth can be assigned proportionally to the calculated load on that link by varying the number of wires in a link or its frequency. In that way the designer calibrates the system resources so that average utilization of all links in the network is approximately equal. At this point, the average load calculation provides only relative link bandwidths. To finalize the design, the QNoC can be simulated and analyzed more precisely by a network simulator. Actual bandwidth can then be assigned to the links according to QoS requirements and the supporting simulation results. Further optimizations can be performed: Buffers and routers can be trimmed where possible while maintaining the required QoS. The entire design process may be iterated if hardware cost of the resulting QNoC is too high, or if other QNoC architectures need to be investigated.It is important to estimate accurately the cost of the QNoC, as it directly influences the cost of the entire system. By having a good measure of cost, the system architect can compare different solutions that provide the same performance and choose the most cost-effective one. We employ area and power cost functions, as is common for VLSI systems, by comparing architectures having the same performance (in terms of delay and throughput) and quantifying their area and power requirements.The cost of QNoC architecture consists of two main factors: the cost of routers and module interfaces (logic cost), and the cost of wires of the links that interconnect them. We assume that the logic and links are designed in an effective way such that power is consumed only when information is flowing through these components (only when logical transitions happen). For power saving, packets traverse the network over the shortest path. In addition, no retransmissions of information are needed since the transmission over a link is assumed to be reliable 3 and hop-by-hop flow control (back pressure) prevents losing or dropping any packet. These facts result in a power efficient network architecture.Wire cost: Since the distance between two adjacent wires is fixed, the area occupied by link wires on a chip is proportional to the total wire length: wires is proportional to wire length and thus wire length is a good estimator of power dissipated on wires. Dynamic power dissipation in switching circuits is:where: L C -load capacitance, dd V -supply voltage and p f -switching frequency.Switching frequency of every link is link frequency multiplied by the link utilization. C L -is the total load capacitance, consisting of wire capacitance (C wire ) and gate capacitance of the transistor driven by that wire (C gate ). We assume that C gate can be neglected and the dominant factor is C wire , which is directly proportional to the length of the wire:where: 0 P -constant coefficient,U -utilization of the links, f i -frequency of the link i.Therefore, total wire length can be a convenient cost metric when comparing between both power and area costs of alternative interconnection architectures.Logic cost: Logic cost consists of the cost of the routers and the cost of network interfaces of the system modules. In all interconnection architectures, a bus or a network, an interface logic must be implemented. It is beyond the scope of this paper to evaluate the exact interface mechanisms that need to be developed. However, it is clear that in a shared media interconnection (like a bus), each module interface must accommodate higher speed bursts of information as compared to a switched architecture and consequently the cost is increased. The cost of router is affected by several parameters: number of ports (#Port), number of service levels (#SL), flit size (FlitSize), buffer size for each service level (BufSize). A good estimation for the area cost of the router is flip-flop count. Generally the cost of a router is very much architecture specific. We give an estimate for the cost of the router in the architecture 3 Or made reliable using error correction 9 that was presented in Section II.4. The number of flip-flops in router is dominated by the flip-flops used for storing data and control information:Total logic cost of QNoC is summation of costs of all routers in the network:IV. DESIGN EXAMPLES As a simple example of QNoC design for a given SoC, let us consider an array of 16 communicating units arranged in a 4X4 mesh. Each unit is connected to the QNoC via a router that is a part of the network (see Figure 8) . The routing mechanism is a symmetric X-Y routing when the destination x-coordinate is greater than source x-coordinate; otherwise, it is Y-X routing. In that way the traffic between each pair of nodes in both directions traverses the network on the same routing path. We assume that links operate at a frequency of 1GHz, which is a reasonable assumption for short links at current technology. We allocate the final link's bandwidth by adding or trimming wires from it. We define flit size to be 16 bits, and use minimal buffering requirements -buffers capable of storing two flits at each input port (so small Signaling or Real-Time packets will not be stretched beyond a single stage and create excessive blocking). The delay of packets in the network consists of the delay of the links and queuing delay in the network. We neglect the delay of routers logic, which should add only a few cycles to the overall delay. During the design process we assign traffic characteristics to the communicating units and simulate the QNoC behavior. OPNET [23] was chosen as a simulation framework for performing this task. OPNET provides a convenient tool for hierarchical modeling of a network, including processes (state machines), network topology description and simulation of different traffic scenarios. The QNoC architecture described in Section II was fully modeled in the OPNET environment and simulated for two scenarios as follows:Each unit communicates uniformly with all other units in the system. Each unit contains four traffic sources that correspond to four classes of system traffic: Signaling, Real-Time, RD/WR and Block-Transfer. Each source creates packets with specific distribution of packet size and packet inter-arrival time. Representative traffic source rates for each service level and its interpretation are summarized in Table 3 . According to that benchmark we obtain average traffic load from each module in the system of about 5.76Gbps, or a total load of 92.16Gbps for the entire 16-modules SoC. This is only one representative example, in our simulations we also checked cases with higher and lower traffic loads to obtain different communication scenarios. 10SLRandom target large block transaction every ~12.5 s.2 000 12 500 2.56Gbps 50 µsec (several times the packet transmission delay on 32 bit, 50 MHz bus) Uniform traffic distribution is unrealistic. More realistic traffic exhibits a non-uniform communication patterns with higher traffic locality. Moreover, according to the proposed design process of the network (Section III), system modules are placed considering their inter-module traffic so as to minimize the system spatial traffic density. In our non-uniform benchmark the network topology and traffic load of the sources is the same as in the uniform-traffic case (Section IV.1, but the probability that a module will send a packet to one of its adjacent neighbors is twice the probability to send the packet to any other module in the network.In order to analyze the results of our benchmarks we define QoS requirements in terms of throughput and packet end-to-end (ETE) delay for each class of service 4 . ETE delay is defined as the sum of the queuing time at the source and travel time through the network experienced by 99% of the packets. The final QNoC configuration must meet those requirements, which are typically defined by the system architect. In our example, we have chosen the maximum ETE delay of a Signaling packet to be no more than 20-30ns, for Real-Time packets we require ETE delay to be the order of magnitude of 125 s, since our Real-Time is voice connection and it shouldn't be more than several frames of a 8KHz clock, and for RD/WR packets we allow ETE delay ~100 nanoseconds. In order to obtain QoS requirements for Block-Transfer packets we consider an alternative solution of a typical system bus that traverses the chip and interconnects all modules on the chip, the bus width is 32 bits and it operates at 50MHz so that its total bandwidth is 1.6Gbps. Just transmission time of one Block-Transfer packet (32000 bits) on such a bus lasts 20 s. Hence we allow ETE delay of a Block-Transfer packet in the QNoC to be no more than several times its transmission time on a typical system bus, see Table 3 .V. OBSERVATIONS AND CONCLUSIONSWe used the design process described in Section IV and applied a uniform traffic load. The modules were placed in a full mesh. Relative traffic load on all the links of the mesh is shown in Figure 9 . Row and column coordinates represent x-y indices of the network routers, for instance point (0,0) corresponds to router_00 at Figure 8 , and bar columns between them represent interrouter links relative load. For example, links (0,0)¡ (1,0) and (2,0)¡ (3,0) have the smallest load in the system, denoted by 1 unit. Other link loads are measured relative to the load on those two links. The highest relative load in the mesh is on link (1,3)¡ (2,3), reaching 9.3. This load distribution originates from traffic distribution and module locations (which are symmetric in our case) and from X-Y coordinates routing, as described in Section IV. 4 Round-trip delay and delay jitter also constitute QoS requirements and may merit future study. Next, link bandwidth was allocated according to the ratios shown in Figure 9 . That allocation led to balanced utilization of the mesh links. We applied the uniform traffic load described in Table 3 (92.16 Gbps) and simulated several total network bandwidth allocation levels. ETE delay was measured at each destination module according to packet service levels. ETE delay was measured in clock cycles of the link (since we assume that links operate at 1GHz, each cycle represents a delay of one nanosecond). The total network bandwidth allocations and obtained results are summarized in Table 4 In the first two cases (Figure 10 and 11) the network is underutilized and delivers better performance than required. By reducing bandwidth (and thus reducing cost) we obtain a network that operates at 30.4% utilization (Figure 12 ). It can be seen that this network configuration delivers the required QoS. Specifically, 99.9% of the Signaling packets arrived with ETE delay of less than 20ns (as required), 99.9% of Real-Time packets arrived with ETE delay of less than 250ns (over-performing, we only required less than 125 s), 99% of RD/WR packets arrived with ETE delay of less than 80ns (as required) and 99.9% of Block-Transfer packets arrived with ETE delay of less than 50 s. That is 2.5 times the transmission time of this packet on an assumed system bus. If we try to reduce the cost any further, the network will not be able to satisfy our QoS requirements as shown in Figure 13 , where requirements for delay of Signaling and Block-Transfer packets are not met.In order to estimate the cost of QNoC systems we use the cost metrics described in Section III. Total wire-length of the links considering data and control wires is ~4m. The cost of the routers is estimated by flip-flop count which results in ~10K flip-flops.Power dissipation is calculated using equation (3) Another important issue is network behavior in terms of delay as a function of traffic load. We chose a fixed network configuration and bandwidth allocation and applied various traffic loads by reducing and expanding packet inter-arrival time for each service level. Figure 14 shows the mean ETE delay of packets at each service level as a function of traffic load in the network. One can observe that while the traffic load is growing, ETE delay of Block-Transfer and RD/WR packets grows exponentially, but the delay of delay-constrained traffic (Real-Time and Signaling) remains nearly constant. Since network resources are kept constant, network utilization grows when higher traffic load is applied (from 16% to 42% in the Figure) . Results for non-uniform traffic are shown in Figure 15 . It can be observed that the ratios between links loads are smaller than in the uniform scenario (maximum link load ratio is 7.25, vs. 9.3 in uniform) and the overall traffic distribution is more balanced because of the higher locality in network traffic. Again, we applied the non-uniform traffic load described in Section IV.2 (92.16 Gbps) and simulated several total network bandwidth allocation levels. ETE delay was measured at each destination module according to packet service levels. The total network bandwidth allocations and obtained results are summarized in Table 5 It can be seen that the network was underutilized in the first two cases (8.2% and 16.5% utilization). Thus we reduced network capacity further, and it can be seen that the network operating at 33.5% utilization (Figure 18 ) was delivering the required QoS. In particular, 99.9% of Signaling packets arrived with ETE delay of less than 20ns (as required), 99.9% of Real-Time packets arrived with ETE delay of less than 270ns, 99.9% of RD/WR packets arrived with ETE delay of less than 150ns and 99% of BlockTransfer packets arrived with ETE delay less than 45 s. That is 2.3 times the transmission time of the same packet on a system bus. If we try to reduce the cost any further, the network will not be able to satisfy our QoS requirements, for example for Signaling and Block-Transfer packets (see Figure 19) .The fact that network traffic in the non-uniform scenario is more local makes it possible to provide the required QoS using less network resources compared with the uniform scenario. Indeed, total wire length of the links considering data and control wires in this case is ~3.5m, compared with 4m in the uniform scenario. This is a 13% reduction in the wire cost of the links. Power dissipation is calculated using equation (3) In this section we compare the cost of QNoC architecture in terms of area and power with the cost of alternative interconnection solutions that provide the same QoS: A shared bus and dedicated point-to-point (PTP) links. We assume a 12×12mm chip comprising sixteen modules.A shared bus in the uniform traffic load design example would have to deliver total traffic load of 92.16 Gbps. Let's also assume that this bus operates at 50 MHz and that it will deliver the required QoS under utilization of 50% (a very optimistic assumption for the given QoS requirements). In order to be competitive with QNoC performance, such a bus would require at least 3700 wires. The bus has to connect to all modules on the chip, and as a result its length would be ~25mm. In practice, shared system buses are multiplexed and there are actually two unidirectional buses. Even if we neglect the significant cost of the multiplexing logic, we obtain a total wire length of ~180m for such a bi-directional bus, as compared with the 4m of the QNoC. Power dissipation on such a bus is calculated using equation (3) We assume that each module is connected to all other modules by dedicated wires. We further assume that point-to-point links operate at 100 MHz. In order to provide the required performance (several times the transmission time of Block-Transfer packet on a system bus), the PTP link should consist of ~6 wires (five data wires and one control wire) and should operate with 80% utilization. Total length of wires that interconnect all 16 modules on chip is ~11.4m. Power dissipation is:, 0 0.9 ptp uniform P P .The comparison of the alternative interconnection architectures for the uniform traffic example is summarized in Figure 21 . It should be noted that the cost of QNoC is several times lower than the cost of bus, both in terms of power dissipation and wire length. The PTP area is also higher than that of the QNoC. Theoretically, a PTP interconnect should consume the same power as the QNoC, because the same traffic is transmitted along the same Manhattan distances and no power is wasted on idle links. However, because of smaller overhead of control wires, the power dissipation of point-to-point solution is slightly lower than in QNoC.For the non-uniform example, the cost of the bus remains the same, because in the bus each transaction is propagated all over the chip and it cannot benefit from higher traffic locality. QNoC cost is reduced (13% reduction in our example) because it benefits directly from traffic locality since less traffic has to be transferred for longer distances. PTP interconnect will also benefit from traffic locality, but its cost remains higher. In this paper we have defined Quality of Service (QoS) and cost model for communications in Systems on Chip (SoCs), and have derived related Network on Chip (NoC) architecture and design process. SoC inter-module communication traffic has been classified into four classes of service: Signaling (for inter-module control signals), Real-Time (for delay-constrained bit streams), RD/WR (for short data access) and Block-Transfer (for large data bursts). The proposed Quality-of-Service NoC (QNoC) design process analyzes the communication traffic of the target SoC and derives QoS requirements (in terms of delay and throughput) for each of the four service classes. A customized QNoC architecture is then created by modifying a generic network architecture. The customization process minimizes the network cost (in area and power) while maintaining the required QoS.. The generic network is based on a two-dimensional planar mesh and fixed shortest path (X-Y based) multi-class wormhole routing. Once communication requirements of the target SoC have been identified, the network is customized as follows: The SoC modules are placed so as to minimize spatial traffic density, unnecessary mesh links and switching nodes are removed, and bandwidth is allocated to the remaining links and switches according to their relative load so that link utilization is balanced. The result is a low cost customized QNoC for the target SoC which guarantees that QoS requirements are met.. A sixteen-module example SoC has been presented and analyzed. We have considered both uniform and non-uniform traffic patterns. In each case we have computed the corresponding QoS requirements, and have demonstrated how an appropriate network (with uniform link utilization but non-uniform bandwidth allocation) can be derived.. The proposed QNoC has also been compared to the shared bus and point-to-point interconnect alternatives. We have shown that the QNoC architecture requires a much shorter total wire length than the two other options, and while being on par with a point-topoint architecture in terms of power, it clearly outperforms shared buses.

B B I O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Communication metrics for software development ABSTRACT † We present empirical evidence that metrics on communication artifacts generated by groupware tools can be used to gain significant insight into the development process that produced them. Moreover, we demonstrate that such communication metrics can provide better insights in the processes and products than metrics based on code.We present a testbed, a software engineering project course, in which we conducted several experiments from 1991 to 1996 with more than 400 participants. Such a testbed provides sufficient realism while allowing for controlled observation of important project parameters. We present three case studies to illustrate the value of communication metrics and propose a statistical framework for validating these hypotheses. Metrics applied to software artifacts have proved to be useful in measuring the impact of a tool or method in the context of a software project. In some cases, they enabled specific problems in the engineering processes to be identified, thus demonstrating the value of metrics and other instrumentation for process improvement. However, software code is only one of the many artifacts produced during software development. Moreover, it is often available only late in the process. The infrastructure and the resources needed to collect metrics on code are non-negligeable, and often prevents their use for identifying problems as they occur.. Communication artifacts, such as electronic mail, memoranda, or records generated by groupware tools, represent a different perspective on the development process. They are available throughout the project, they capture information about a more comprehensive set of artifacts (e.g., code, process, organization, politics, morale), and their form is independent of implementation technology, development infrastructure, or even the existence of a product. Metrics applied to such communication artifacts could then provide significant insight into the development process that produced them.. In this paper, we discuss the design and evaluation of communication metrics for software development. We present a testbed, a senior level software engineering project course at Carnegie Mellon University, in which we conducted several studies and experiments. Such a testbed represents an ideal environment for empirical software engineering, providing sufficient realism while allowing for controlled observation of important project parameters, such as development methodology, communication infrastructure, implementation language. Our testbed is described in the section entitled "A TESTBED FOR COMMUNICATION METRICS".. Then, we present three proof-of-concept experiments to illustrate the value of communication metrics and their possible use (see section entitled "CASE STUDIES"). Measures are defined on the electronic messages exchanged between teams and different patterns in the traffic are associated with specific crisis and differences in outcome.. Finally, in "TOWARDS AN EMPIRICAL FRAMEWORK", we propose a statistical framework for validating specific communication metrics. We instantiate it using data from our testbed in the form of metrics on the bulletin board traffic and on the outcome (software and documents) of a number of projects. Empirical evidence strongly suggests that: 1. under certain conditions, the electronic trafficwas representative enough of the overall communication to provide substantial insight into the development process, 2. using communication metrics instead of outcome metrics is a valid approach.. We conclude this paper by outlining future work necessary for empirical validation of communication metrics. We have estimated two structural equation models for explaining the outcome size as well as the outcome complexity. For both models we used data from 6 testbed projects (Fall'91, '92,'93, and '94, Spring'94 and '95; the Spring'91, Fall '95, and Spring '96 projects have not yet been included in the model). Both models are based on the general model shown in Figure 8 . The size and complexity of source code was measured using Halstead volume and size metrics [12] . The size and complexity of documents and Bboard messages were measured in terms of number of words and number of unique noun phrases. Noun phrases were counted using off-the-shelf natural language processing tools [6] .. For the two models we considered 14 potential sources of variation and modeled them with 27 independent variables. Some sources of variation are modeled as multiple independent variables, e.g., the development methodology used in the testbed project was modeled with the 3 independent variables X OMT , X OOSE , and X Hybrid .. In the size model, 6 independent variables had a statistically significant impact on outcome, while 9 had an impact on communication (indirect effects). The communication size variable had also a significant impact on outcome. In the complexity model, 10 independent variables had an impact on outcome, while 11 independent variables had an impact on communication. Given that space does not allow a thorough discussion of the estimation of these models, we only present here a brief summary of these results. The reader is referred to [10] for a complete presentation of the methods and results of these structural models. Table 2 displays the estimates of the direct effects of selected independent variables for the size and complexity models. OMT represented by the parameter γ OMT had a strong impact (3.7656) on the size of the deliverables modeled by h, which can be explained in part by the code generation facilities available in OMTool. OMT was unfortunately also responsible for an increase in deliverable complexity (0.4798). We interpret that result as an increase in productivity and in the amount of redundancy in deliverables. itIWEB represented by the parameter γ itIWEB also contributed (0.7346) to an increase in size of the deliverables while not increasing the complexity of deliverables. We interpret this result as an improvement in the productivity of students. The number of liaisons per team represented by γ n_liaisons reduced the complexity of deliverables which can be interpreted as a better communication structure. That is, better communication leads to a faster convergence of participants on common concepts and terms, thus reducing complexity. Finally, the last row in Table 2 represents θ comm , the contribution of Bboard communication on deliverable size and complexity. As we can see, an increase in communication results in larger and less complex deliverables. Table 3 displays the estimates of the indirect effects of selected independent variables through Bboard communications. OMT reduced the size and complexity of Bboard communication, which is due to an increase communication of developers through an alternate channel, OMTool. The reduction of the size and complexity of Bboard communication due to itIWEB is due to a decrease in spontaneous Bboard communication (i.e., the data presented in the previous section indicated that the traffic associated with agendas and minutes actually increased). We interpret this as an increase of communication associated with the meeting, thus reducing the need for spontaneous, crisisdriven interactions. Finally, note that Bboard communication is a function of the number of developers and the square number of developers, as expected.. Note that the approach we adopted in building this statistical framework is that of quasi-experimental design. Although we do not have the full control over the experimental context that would enable us a true or classical experiment, the use of statistical procedures is still valuable for modeling sources of variations at hand. This is also consistent with our formative approach, in which we feedback into the testbed lessons learned during case studies and statistical analysis.

B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O