Abstract: In this paper we compare two approaches to the design of protocol frameworks -- tools for implementing modular network protocols. The most common approach uses events as the main abstraction for a local interaction between protocol modules. We argue that an alternative approach, that is based on service abstraction, is more suitable for expressing modular protocols. It also facilitates advanced features in the design of protocols, such as dynamic update of distributed protocols. We then describe an experimental implementation of a service-based protocol framework in Java.

Service Interface: A New Abstraction for Implementing and Composing Protocols∗ Olivier R¨utti Paweł T. Wojciechowski Andr´e Schiper Ecole Polytechnique F´ed´erale de Lausanne (EPFL) 1015 Lausanne, Switzerland {Olivier.Rutti, Pawel.Wojciechowski, Andre.Schiper}@epfl.
ch ABSTRACT In this paper we compare two approaches to the design of protocol frameworks - tools for implementing modular network protocols.
The most common approach uses events as the main abstraction for a local interaction between protocol modules.
We argue that an alternative approach, that is based on service abstraction, is more suitable for expressing modular protocols.
It also facilitates advanced features in the design of protocols, such as dynamic update of distributed protocols.
We then describe an experimental implementation of a service-based protocol framework in Java.
Categories and Subject Descriptors C.2.4 [Computer-Communication Networks]: Distributed Applications 1.
INTRODUCTION Protocol frameworks, such Cactus [5, 2], Appia [1, 16], Ensemble [12, 17], Eva [3], SDL [8] and Neko[6, 20], are programming tools for developing modular network protocols.
They allow complex protocols to be implemented by decomposing them into several modules cooperating together.
This approach facilitates code reuse and customization of distributed protocols in order to fit the needs of different applications.
Moreover, protocol modules can be plugged in to the system dynamically.
All these features of protocol frameworks make them an interesting enabling technology for implementing adaptable systems [14] - an important class of applications.
Most protocol frameworks are based on events (all frameworks cited above are based on this abstraction).
Events are used for asynchronous communication between different modules on the same machine.
However, the use of events raises some problems [4, 13].
For instance, the composition of modules may require connectors to route events, which introduces burden for a protocol composer [4].
Protocol frameworks such as Appia and Eva extend the event-based approach with channels.
However, in our opinion, this solution is not satisfactory since composition of complex protocol stacks becomes more difficult.
In this paper, we propose a new approach for building modular protocols, that is based on a service abstraction.
We compare this new approach with the common, event-based approach.
We show that protocol frameworks based on services have several advantages, e.g. allow for a fairly straightforward protocol composition, clear implementation, and better support of dynamic replacement of distributed protocols.
To validate our claims, we have implemented SAMOA - an experimental protocol framework that is purely based on the service-based approach to module composition and implementation.
The framework allowed us to compare the service- and event-based implementations of an adaptive group communication middleware.
The paper is organized as follows.
Section 2 defines general notions.
Section 3 presents the main characteristics of event-based frameworks, and features that are distinct for each framework.
Section 4 describes our new approach, which is based on service abstraction.
Section 5 discusses the advantages of a service-based protocol framework compared to an event-based protocol framework.
The description of our experimental implementation is presented in Section 6.
Finally, we conclude in Section 7.
2.
PROTOCOL FRAMEWORKS In this section, we describe notions that are common to all protocol frameworks.
Protocols and Protocol Modules.
A protocol is a distributed algorithm that solves a specific problem in a distributed system, e.g. a TCP protocol solves the reliable channel problem.
A protocol is implemented as a set of identical protocol modules located on different machines.
Protocol Stacks.
A stack is a set of protocol modules (of different protocols) that are located on the same machine.
Note that, despite its name, a stack is not strictly layered, 691 i.e. a protocol module can interact with all other protocol modules in the same stack, not only with the protocol modules directly above and below.
In the remainder of this paper, we use the terms machine and stack interchangeably.
Stack 1 S1 Q1 R1 P1 Network Figure 1: Example of a protocol stack In Figure 1, we show an example protocol stack.
We represent protocol modules by capital letters indexed with a natural number, e.g. P1, Q1, R1 and S1.
We write Pi to denote the protocol module of a protocol P in stack i.
We use this notation throughout the paper.
Modules are represented as white boxes.
Arrows show module interactions.
For instance, protocol module P1 interacts with the protocol module Q1 and conversely (See Fig. 1).
Protocol Module Interactions.
Below, we define the different kinds of interaction between protocol modules.
• Requests are issued by protocol modules.
A request by a protocol module Pi is an asynchronous call by Pi of another protocol module.
• Replies are the results of a request.
A single request can generate several replies.
Only protocol modules belonging to the same protocol as the module that has issued the request are concerned by the corresponding replies.
For example, a request by Pi generates replies that concern only protocol modules Pj.
• Notifications can be used by a protocol module to inform (possibly many) protocol modules in the same stack about the occurrence of a specific event.
Notifications may also be the results of a request.
3.
EVENT-BASED PROTOCOL FRAMEWORK DESIGN Most existing protocol frameworks are event-based.
Examples are Cactus [5, 2], Appia [1, 16] and Ensemble [12, 17].
In this section, we define the notion of an event in protocol frameworks.
We also explain how protocol modules are structured in event-based frameworks.
Events.
An event is a special object for indirect communication between protocol modules in the same stack.
Events may transport some information, e.g. a network message or some other data.
With events, the communication is indirect, i.e. a protocol module that triggers an event is not aware of the module(s) that handle the event.
Events enable one-to-many communication within a protocol stack.
Triggering an event can be done either synchronously or asynchronously.
In the former case, the thread that triggers an event e is blocked until all protocol modules that handle e have terminated handling of event e.
In the latter case, the thread that triggers the event is not blocked.
Protocol Modules.
In event-based protocol frameworks, a protocol module consists of a set of handlers.
Each handler is dedicated to handling of a specific event.
Handlers of the same protocol module may share data.
Handlers can be dynamically bound to events.
Handlers can also be unbound dynamically.
Upon triggering some event e, all handlers bound to e are executed.
If no handler is bound, the behavior is usually unspecified.
Stack 1 P1 Q1 R1 S1 Network f e gg deliver send h Figure 2: Example of an event-based protocol stack In Figure 2, we show an example of an event-based stack.
Events are represented by small letters, e.g. e, f, ... The fact that a protocol module can trigger an event is represented by an arrow starting from the module.
A white trapezoid inside a module box represents a handler defined by the protocol module.
To mark that some handler is bound to event e, we use an arrow pointing to the handler (the label on the arrow represents the event e).
For example, the protocol module P1 triggers event e and handles event f (see Fig. 2).
Note that the network is represented as a special protocol module that handles the send event (to send a message to another machine) and triggers the deliver event (upon receipt of a message from another machine).
Specific Features.
Some protocol frameworks have unique features.
Below, we present the features that influence composition and implementation of protocol modules.
In Cactus [5, 2], the programmer can give a priority number to a handler upon binding it to an event.
When an event is triggered, all handlers are executed following the order of priority.
A handler h is also able to cancel the execution of an event trigger: all handlers that should be executed after h according to the priority are not executed.
Appia [1, 16] and Eva [3] introduce the notion of channels.
Channels allow to build routes of events in protocol stacks.
Each protocol module has to subscribe to one or many channels.
All events are triggered by specifying a channel they belong to.
When a protocol module triggers an event e specifying channel c, all handlers bound to e that are part of a protocol that subscribes to c are executed (in the order prescribed by the definition of channel c).
4.
SERVICE-BASED PROTOCOL FRAMEWORK In this section, we describe our new approach for implementing and composing protocols that is based on services.
692 We show in Section 5 the advantages of service-based protocol frameworks over event-based protocol frameworks.
Service Interface.
In our service-based framework, protocol modules in the same stack communicate through objects called service interfaces.
Requests, replies and notifications are all issued to service interfaces.
Protocol Modules.
A protocol module is a set of executers, listeners and interceptors.
Executers handle requests.
An executer can be dynamically bound to a service interface.
It can be later unbound.
A request issued to a service interface si leads to the execution of the executer bound to si.
If no executer is bound to si, the request is delayed until some executer is bound to si.
Contrary to events, at most one executer at any time can be bound to a service interface on every machine.
Listeners handle replies and notifications.
A listener can be dynamically bound and unbound to/from a service interface si.
A notification issued to a service interface si is handled by all listeners bound to si in the local stack.
A reply issued to a service interface is handled by one single listener.
To ensure that one single listener handles a reply, a module Pi has to identify, each time it issues a request, the listener to handle the possible reply.
If the request and the reply occur respectively, in stack i and in stack j, the service interface si on i communicates to the service interface si on j the listener that must handle the reply.
If the listener that must handle the reply does not exist, the reply is delayed until the listener is created.
Stack 1 P1 Q1 R1 S1 Network t u nt Figure 3: Example of a service-based protocol stack In Figure 3, we show an example of a service-based stack.
We denote a service interface by a small letter (e.g. t, u and nt) in a hexagonal box.
The fact that a module Pi can generate a request to a service interface si is represented by a dashed black arrow going from Pi to si.
Similarly, a dashed white arrow going from module Pi to service interface si represents the fact that Pi can generate a reply or a notification to si.
We represent executers with white boxes inside protocol modules and listeners with white boxes with a gray border.
A connecting line between a service interface si and an executer e (resp.
a listener l) shows that e (resp.
l) is bound to si.
In Figure 3, module Q1 contains an executer bound to service interface t and a listener bound to service interface u. Module Q1 can generate replies and notifications to service interface t and requests to service interface u. Note that the service interface nt allows to access the network.
P1 Q1 P1 Q1 T1T1 t t t Figure 4: Execution of protocol interactions with interceptors An interceptor plays a special rˆole.
Similarly to executers, interceptors can be dynamically bound or unbound to a service interface.
They are activated each time a request, a reply or a notification is issued to the service interface they are bound to.
This is illustrated in Figure 4.
In the right part of the figure, the interceptor of the protocol module T1 is represented by a rounded box.
The interceptor is bound to service interface t.
The left part of the figure shows that an interceptor can be seen as an executer plus a listener.
When P1 issues a request req to the service interface t, the executer-interceptor of T1 is executed.
Then, module T1 may forward a request req to the service interface t, where we can have req = req 1 .
When module Q1 issues a reply or a notification, a similar mechanism is used, except that this time the listener-interceptor of T1 is executed.
Note that a protocol module Ti, that has an interceptor bound to a service interface, is able to modify requests, replies and notifications.
Upon requests, if several interceptors are bound to the same service interface, they are executed in the order of binding.
Upon replies and notifications, the order is reversed.
5.
ADVANTAGES OF SERVICE-BASED PROTOCOL FRAMEWORK DESIGN We show in this section the advantages of service-based protocol frameworks over event-based protocol frameworks.
We structure our discussion in three parts.
Firstly, we present how protocol interactions are modeled in each of the protocol frameworks.
Then, we discuss the composition of protocol modules in each of these frameworks.
Finally, we present the problem of dynamic protocol replacement and the advantages of service interfaces in order to implement it.
The discussion is summarized in Table 1.
5.1 Protocol Module Interactions A natural model of protocol interactions (as presented in Section 2) facilitates the implementation of protocol modules.
For each protocol interaction, we show how it is modeled in both frameworks.
We also explain that an inadequate model may lead to problems.
Requests.
In service-based frameworks, a request is generated to a service interface.
Each request is handled by at most one executer, since we allow only one executer to be bound to a service interface at any time.
On the other hand, in event-based frameworks, a protocol module emulates a request by triggering an event.
There is no guarantee 1 The two service interfaces t in the left part of Figure 4 represent the same service interface t.
The duplication is only to make the figure readable.
693 that this event is bound to only one handler, which may lead to programming errors.
Replies.
When a protocol module generates a reply in a service-based framework, only the correct listener (identified at the time the corresponding request was issued) is executed.
This ensures that a request issued by some protocol module Qi, leads to replies handled by protocol modules Qj (i.e. protocol modules of the same protocol).
This is not the case in event-based frameworks, as we now show.
Consider protocol module Q1 in Figure 2 that triggers event g to emulate a request.
Module S1 handles the request.
When modules Si triggers event h to emulate a reply (remember that a reply can occur in many stacks), both modules Qi and Ri will handle the reply (they both contain a handler bound to h).
This behavior is not correct: only protocol modules Qi should handle the reply.
Moreover, as modules Ri are not necessarily implemented to interact with modules Qi, this behavior may lead to errors.
Solutions to solve this problem exist.
However, they introduce an unnecessary burden on the protocol programmers and the stack composer.
For instance, channels allow to route events to ensure that modules handle only events concerning them.
However, the protocol programmer must take channels into account when implementing protocols.
Moreover, the composition of complex stacks becomes more difficult due to the fact that the composer has to create many channels to ensure that modules handle events correctly.
An addition of special protocol modules (named connectors) for routing events is also not satisfactory, since it requires additional work from the composer and introduces overhead.
Notifications.
Contrary to requests and replies, notifications are well modeled in event-based frameworks.
The reason is that notifications correspond to the one-to-many communication scheme provided by events.
In service-based frameworks, notifications are also well modeled.
When a module generates a notification to a service interface si, all listeners bound to s are executed.
Note that in this case, service interfaces provide the same pattern of communication as events.
5.2 Protocol Module Composition Replies (and sometimes notifications) are the results of a request.
Thus, there is a semantic link between them.
The composer of protocol modules must preserve this link in order to compose correct stacks.
We explain now that service based frameworks provide a mechanism to preserve this link, while in event-based frameworks, the lack of such mechanism leads to error-prone composition.
In service-based frameworks, requests, replies and notifications are issued to a service interface.
Thus, a service interface introduces a link between these interactions.
To compose a correct stack, the composer has to bound a listener to service interface si for each module that issues a request to si.
The same must be done for one executer that is part of a module that issues replies or notifications.
Applying this simple methodology ensures that every request issued to a service interface si eventually results in several replies or notifications issued to the same service interface si.
In event-based frameworks, all protocol interactions are issued through different events: there is no explicit link between an event triggered upon requests and an event triggered upon the corresponding replies.
Thus, the composer of a protocol stack must know the meaning of each event in order to preserve the semantic link between replies (and notifications) and requests.
Moreover, nothing prevents from binding a handler that should handle a request to an event used to issue a reply.
Note that these problems can be partially solved by typing events and handlers.
However, it does not prevent from errors if there are several instances of the same event type.
Note that protocol composition is clearer in the protocol frameworks that are based on services, rather than on events.
The reason is that several events that are used to model different protocol interactions can be modeled by a single service interface.
5.3 Dynamic Replacement of Protocols Dynamic replacement of protocols consists in switching on-the-fly between protocols that solve the same problem.
Replacement of a protocol P by a new protocol newP means that a protocol module Pi is replaced by newPi in every stack i.
This replacement is problematic since the local replacements (within stacks) must be synchronized in order to guarantee protocol correctness [21, 18].
Q1 Q1 R1 P1 1P 1newP 1 Repl−P1 Repl−P1 R newP1 gg h h'' g'' t Figure 5: Dynamic replacement of protocol P For the synchronization algorithms to work, module interactions are intercepted in order to detect a time when Pi should be replaced by newPi.
(Other solutions, e.g. in [11], are more complex.)
In Fig. 5, we show how this interception can be implemented in protocol frameworks that are based on services (in the left part of the figure) and events (in the right part of the figure).
The two-sided arrows point to the protocol modules P1 and newP1 that are switched.
It can be seen that the approach that uses the Service Interface mechanism has advantages.
The intercepting module Repl-P1 has an interceptor bound to service interface t that intercepts every request handled by modules P1 and all replies and notifications issued by P1.
The code of the module P1 can therefore remain unchanged.
In event-based frameworks, the solution is to add an intermediate module Repl-P1 that intercepts the requests issued to P1 and also the replies and notifications issued by P1.
Although this ad-hoc solution may seem similar to the servicebased approach, there is an important difference.
The eventbased solution requires to slightly modify the module P1 since instead of handling event g and triggering event h, P1 must now handle different events g'' and h'' (see Fig. 5).
6.
IMPLEMENTATION We have implemented an experimental service-based protocol framework (called SAMOA) [7].
Our implementation is light-weight: it consists of approximately 1200 lines of code in Java 1.5 (with generics).
In this section, we describe the main two classes of our implementation: Service (encoding the Service Interface) and 694 service-based event-based Protocol Interaction an adequate an inadequate representation representation Protocol Composition clear and safe complex and error-prone Dynamic Replacement an integrated ad-hoc solutions mechanism Table 1: Service-based vs. event-based Protocol (encoding protocol modules).
Finally, we present an example protocol stack that we have implemented to validate the service-based approach.
The Service Class.
A Service object is characterized by the arguments of requests and the arguments of responses.
A response is either a reply or a notification.
A special argument, called message, determines the kind of interactions modeled by the response.
A message represents a piece of information sent over the network.
When a protocol module issues a request, it can give a message as an argument.
The message can specify the listener that must handle the reply.
When a protocol module issues a response to a service interface, a reply is issued if one of the arguments of the response is a message specifying a listener.
Otherwise, a notification is issued.
Executers, listeners and interceptors are encoded as innerclasses of the Service class.
This allows to provide type-safe protocol interactions.
For instance, executers can only be bound to the Service object, they belong to.
Thus, the parameters passed to requests (that are verified statically) always correspond to the parameters accepted by the corresponding executers.
The type of a Service object is determined by the type of the arguments of requests and responses.
A Service object t is compatible with another Service object s if the type of the arguments of requests (and responses) of t is a subtype of the arguments of requests (and responses) of s.
In practice, if a protocol module Pi can issue a request to a protocol UDP, then it may also issue a request to TCP (compatible with UDP) due to the subtyping relation on parameters of communicating modules.
The Protocol Class.
A Protocol object consists of three sets of components, one set for each component type (a listener, an executer, and an interceptor).
Protocol objects are characterized by names to retrieve them easily.
Moreover, we have added some features to bind and unbind all executers or interceptors to/from the corresponding Service objects.
Protocol objects can be loaded to a stack dynamically.
All these features made it easy to implement dynamic replacement of network protocols.
Protocol Stack Implementation.
To validate our ideas, we have developed an Adaptive Group Communication (AGC) middleware, adopting both the service- and the event-based approaches.
Fig. 6 shows the corresponding stacks of the AGC middleware.
Both stacks allow the Consensus and Atomic Broadcast protocols to be dynamically updated.
The architecture of our middleware, shown in Fig. 6, builds on the group communication stack described in [15].
The UDP and RP2P modules provide respectively, unreliable and reliable point-to-point transport.
The FD module implements a failure detector; we assume that it ensures the Stack 1 UDP1RP2P1 Repl CT1 1ABc.
Repl CT1 ABc.1 Network FD1 GM1 rp2p nt udp d f abcast consensus Stack 1 Repl CT1 1ABc.
Repl ABc.1 UDP1 FD1 RP2P1 CT1 Network 1GM send deliver Figure 6: Adaptive Group Communication Middleware: service-based (left) vs. event-based (right) properties of the 3S failure detector [9].
The CT module provides a distributed consensus service using the ChandraToueg algorithm [10].
The ABc.
module implements atomic broadcast - a group communication primitive that delivers messages to all processes in the same order.
The GM module provides a group membership service that maintains consistent membership data among group members (see [19] for details).
The Repl ABc.
and the Repl CT modules implement the replacement algorithms [18] for, respectively, the ABc.
and the CT protocol modules.
Note that each arrow in the event-based architecture represents an event.
We do not name events in the figure for readability.
The left stack in Figure 6 shows the implementation of AGC with our service-based framework.
The right stack shows the same implementation with an event-based framework.
Performance Evaluation.
To evaluate the overhead of service interfaces, we compared performance of the serviceand event-based implementations of the AGC middleware.
The latter implementation of AGC uses the Cactus protocol framework [5, 2].
In our experiment, we compared the average latency of Atomic Broadcast (ABcast), which is defined as follows.
Consider a message m sent using ABcast.
We denote by ti(m) the time between the moment of sending m and the moment of delivering m on a machine (stack) i.
We define the average latency of m as the average of ti(m) for all machines (stacks) i within a group of stacks.
Performance tests have been made using a cluster of PCs running Red Hat Linux 7.2, where each PC has a Pentium III 766 MHz processor and 128MB of RAM.
All PCs are interconnected by a 100 Base-TX duplex Ethernet hub.
Our experiment has involved 7 machines (stacks) that ABcast messages of 4Mb under a constant load, where a load is a number of messages per second.
In Figure 7, we show the results of our experiment for different loads.
Latencies are shown on the vertical axis, while message loads are shown on the horizontal axis.
The solid line shows the results obtained with our service-based framework.
The dashed line shows the results obtained with the Cactus framework.
The 695 0 500 1000 1500 2000 10 20 30 40 50 60 70 80 90 100 Averagelatency[ms] Load [msg/s] Service-Based Framework Cactus Figure 7: Comparison between our service-based framework and Cactus overhead of the service-based framework is approximately 10%.
This can be explained as follows.
Firstly, the servicebased framework provides a higher level abstraction, which has a small cost.
Secondly, the AGC middleware was initially implemented and optimized for the event-based Cactus framework.
However, it is possible to optimize the AGC middleware for the service-based framework.
7.
CONCLUSION In the paper, we proposed a new approach to the protocol composition that is based on the notion of Service Interface, instead of events.
We believe that the service-based framework has several advantages over event-based frameworks.
It allows us to: (1) model accurately protocol interactions, (2) reduce the risk of errors during the composition phase, and (3) simply implement dynamic protocol updates.
A prototype implementation allowed us to validate our ideas.
8.
REFERENCES [1] The Appia project.
Documentation available electronically at http://appia.di.fc.ul.pt/.
[2] Nina T. Bhatti, Matti A. Hiltunen, Richard D. Schlichting, and Wanda Chiu.
Coyote: a system for constructing fine-grain configurable communication services.
ACM Transactions on Computer Systems, 16(4):321-366, November 1998.
[3] Francisco Vilar Brasileiro, Fab´ıola Greve, Frederic Tronel, Michel Hurfin, and Jean-Pierre Le Narzul.
Eva: An event-based framework for developing specialized communication protocols.
In Proceedings of the 1st IEEE International Symposium on Network Computing and Applications (NCA ``01), 2001.
[4] Daniel C. B¨unzli, Sergio Mena, and Uwe Nestmann.
Protocol composition frameworks.
A header-driven model.
In Proceedings of the 4th IEEE International Symposium on Network Computing and Applications (NCA ``05), July 2005.
[5] The Cactus project.
Documentation available electronically at http://www.cs.arizona.edu/ cactus/.
[6] The Neko project.
Documentation available electronically at http://lsrwww.epfl.ch/neko/.
[7] The SAMOA project.
Documentation available electronically at http://lsrwww.epfl.ch/samoa/.
[8] The SDL project.
Documentation available electronically at http://www.sdl-forum.org/SDL/.
[9] Tushar Deepak Chandra, Vassos Hadzilacos, and Sam Toueg.
The weakest failure detector for solving consensus.
Journal of the ACM, 43(4):685-722, 1996.
[10] Tushar Deepak Chandra and Sam Toueg.
Unreliable failure detectors for reliable distributed systems.
Journal of the ACM, 43(2):225-267, 1996.
[11] Wen-Ke Chen, Matti A. Hiltunen, and Richard D. Schlichting.
Constructing adaptive software in distributed systems.
In Proceedings of the 21st IEEE International Conference on Distributed Computing System (ICDCS ``01), April 2001.
[12] The Ensemble project.
Documentation available electronically at http://www.cs.cornell.edu/Info/ Projects/Ensemble/.
[13] Richard Ekwall, Sergio Mena, Stefan Pleisch, and Andr´e Schiper.
Towards flexible finite-state-machine-based protocol composition.
In Proceedings of the 3rd IEEE International Symposium on Network Computing and Applications (NCA ``04), August 2004.
[14] Philip K. McKinley, Seyed Masoud Sadjadi, Eric P. Kasten, and Betty H.C. Cheng.
Composing adaptive software.
IEEE Computer, 37(7):56-64, 2004.
[15] Sergio Mena, Andr´e Schiper, and Pawel T. Wojciechowski.
A step towards a new generation of group communication systems.
In Proceedings of the 4th ACM/IFIP/USENIX International Middleware Conference (Middleware ``03), LNCS 2672, June 2003.
[16] Hugo Miranda, Alexandre Pinto, and Lu´ıs Rodrigues.
Appia, a flexible protocol kernel supporting multiple coordinated channels.
In Proceedings of the 21st IEEE International Conference on Distributed Computing Systems (ICDCS ``01), April 2001.
[17] Ohad Rodeh, Kenneth P. Birman, Mark Hayden, Zhen Xiao, and Danny Dolev.
The architecture and performance of security protocols in the Ensemble group communication system.
Technical Report TR-98-1703, Computer Science Department, Cornell University, September 1998.
[18] Olivier R¨utti, Pawel T. Wojciechowski, and Andr´e Schiper.
Dynamic update of distributed agreement protocols.
TR IC-2005-12, School of Computer and Communication Sciences, Ecole Polytechnique F´ed´erale de Lausanne (EPFL), March 2005.
[19] Andr´e Schiper.
Dynamic Group Communication.
Technical Report IC-2003-27, School of Computer and Communication Sciences, Ecole Polytechnique F´ed´erale de Lausanne (EPFL), April 2003.
To appear in ACM Distributed Computing.
[20] P´eter Urb´an, Xavier D´efago, and Andr´e Schiper.
Neko: A single environment to simulate and prototype distributed algorithms.
In Proceedings of the 15th International Conference on Information Networking (ICOIN ``01), February 2001.
[21] Pawel T. Wojciechowski and Olivier R¨utti.
On correctness of dynamic protocol update.
In Proceedings of the 7th IFIP Conference on Formal Methods for Open Object-Based Distributed Systems (FMOODS ``05), LNCS 3535.
Springer, June 2005.
696

Service Interface: A New Abstraction for Implementing and Composing Protocols *
ABSTRACT
In this paper we compare two approaches to the design of protocol frameworks--tools for implementing modular network protocols.
The most common approach uses events as the main abstraction for a local interaction between protocol modules.
We argue that an alternative approach, that is based on service abstraction, is more suitable for expressing modular protocols.
It also facilitates advanced features in the design of protocols, such as dynamic update of distributed protocols.
We then describe an experimental implementation of a service-based protocol framework in Java.
1.
INTRODUCTION
Protocol frameworks, such Cactus [5, 2], Appia [1, 16], Ensemble [12, 17], Eva [3], SDL [8] and Neko [6, 20], are programming tools for developing modular network protocols.
They allow complex protocols to be implemented by decomposing them into several modules cooperating together.
This approach facilitates code reuse and customization of distributed protocols in order to fit the needs of different applications.
Moreover, protocol modules can be plugged in to the system dynamically.
All these features of protocol frameworks make them an interesting enabling technology for implementing adaptable systems [14] - an important class of applications.
* Research supported by the Swiss National Science Foundation under grant number 21-67715 .02 and Hasler Stiftung under grant number DICS-1825.
Most protocol frameworks are based on events (all frameworks cited above are based on this abstraction).
Events are used for asynchronous communication between different modules on the same machine.
However, the use of events raises some problems [4, 13].
For instance, the composition of modules may require connectors to route events, which introduces burden for a protocol composer [4].
Protocol frameworks such as Appia and Eva extend the event-based approach with channels.
However, in our opinion, this solution is not satisfactory since composition of complex protocol stacks becomes more difficult.
In this paper, we propose a new approach for building modular protocols, that is based on a service abstraction.
We compare this new approach with the common, event-based approach.
We show that protocol frameworks based on services have several advantages, e.g. allow for a fairly straightforward protocol composition, clear implementation, and better support of dynamic replacement of distributed protocols.
To validate our claims, we have implemented SAMOA--an experimental protocol framework that is purely based on the service-based approach to module composition and implementation.
The framework allowed us to compare the service - and event-based implementations of an adaptive group communication middleware.
The paper is organized as follows.
Section 2 defines general notions.
Section 3 presents the main characteristics of event-based frameworks, and features that are distinct for each framework.
Section 4 describes our new approach, which is based on service abstraction.
Section 5 discusses the advantages of a service-based protocol framework compared to an event-based protocol framework.
The description of our experimental implementation is presented in Section 6.
Finally, we conclude in Section 7.
2.
PROTOCOL FRAMEWORKS
3.
EVENT-BASED PROTOCOL FRAMEWORK DESIGN
4.
SERVICE-BASED PROTOCOL FRAMEWORK
5.
ADVANTAGES OF SERVICE-BASED PROTOCOL FRAMEWORK DESIGN
5.1 Protocol Module Interactions
5.2 Protocol Module Composition
5.3 Dynamic Replacement of Protocols
6.
IMPLEMENTATION
7.
CONCLUSION
In the paper, we proposed a new approach to the protocol composition that is based on the notion of Service Interface, instead of events.
We believe that the service-based framework has several advantages over event-based frameworks.
It allows us to: (1) model accurately protocol interactions, (2) reduce the risk of errors during the composition phase, and (3) simply implement dynamic protocol updates.
A prototype implementation allowed us to validate our ideas.

Service Interface: A New Abstraction for Implementing and Composing Protocols *
ABSTRACT
In this paper we compare two approaches to the design of protocol frameworks--tools for implementing modular network protocols.
The most common approach uses events as the main abstraction for a local interaction between protocol modules.
We argue that an alternative approach, that is based on service abstraction, is more suitable for expressing modular protocols.
It also facilitates advanced features in the design of protocols, such as dynamic update of distributed protocols.
We then describe an experimental implementation of a service-based protocol framework in Java.
1.
INTRODUCTION
They allow complex protocols to be implemented by decomposing them into several modules cooperating together.
This approach facilitates code reuse and customization of distributed protocols in order to fit the needs of different applications.
Moreover, protocol modules can be plugged in to the system dynamically.
All these features of protocol frameworks make them an interesting enabling technology for implementing adaptable systems [14] - an important class of applications.
Most protocol frameworks are based on events (all frameworks cited above are based on this abstraction).
Events are used for asynchronous communication between different modules on the same machine.
For instance, the composition of modules may require connectors to route events, which introduces burden for a protocol composer [4].
Protocol frameworks such as Appia and Eva extend the event-based approach with channels.
However, in our opinion, this solution is not satisfactory since composition of complex protocol stacks becomes more difficult.
In this paper, we propose a new approach for building modular protocols, that is based on a service abstraction.
We compare this new approach with the common, event-based approach.
We show that protocol frameworks based on services have several advantages, e.g. allow for a fairly straightforward protocol composition, clear implementation, and better support of dynamic replacement of distributed protocols.
To validate our claims, we have implemented SAMOA--an experimental protocol framework that is purely based on the service-based approach to module composition and implementation.
The framework allowed us to compare the service - and event-based implementations of an adaptive group communication middleware.
Section 2 defines general notions.
Section 3 presents the main characteristics of event-based frameworks, and features that are distinct for each framework.
Section 4 describes our new approach, which is based on service abstraction.
Section 5 discusses the advantages of a service-based protocol framework compared to an event-based protocol framework.
The description of our experimental implementation is presented in Section 6.
Finally, we conclude in Section 7.
7.
CONCLUSION
In the paper, we proposed a new approach to the protocol composition that is based on the notion of Service Interface, instead of events.
We believe that the service-based framework has several advantages over event-based frameworks.
A prototype implementation allowed us to validate our ideas.

Service Interface: A New Abstraction for Implementing and Composing Protocols *
ABSTRACT
In this paper we compare two approaches to the design of protocol frameworks--tools for implementing modular network protocols.
The most common approach uses events as the main abstraction for a local interaction between protocol modules.
We argue that an alternative approach, that is based on service abstraction, is more suitable for expressing modular protocols.
It also facilitates advanced features in the design of protocols, such as dynamic update of distributed protocols.
We then describe an experimental implementation of a service-based protocol framework in Java.
1.
INTRODUCTION
Protocol frameworks, such Cactus [5, 2], Appia [1, 16], Ensemble [12, 17], Eva [3], SDL [8] and Neko [6, 20], are programming tools for developing modular network protocols.
They allow complex protocols to be implemented by decomposing them into several modules cooperating together.
This approach facilitates code reuse and customization of distributed protocols in order to fit the needs of different applications.
Moreover, protocol modules can be plugged in to the system dynamically.
All these features of protocol frameworks make them an interesting enabling technology for implementing adaptable systems [14] - an important class of applications.
* Research supported by the Swiss National Science Foundation under grant number 21-67715 .02 and Hasler Stiftung under grant number DICS-1825.
Most protocol frameworks are based on events (all frameworks cited above are based on this abstraction).
Events are used for asynchronous communication between different modules on the same machine.
However, the use of events raises some problems [4, 13].
For instance, the composition of modules may require connectors to route events, which introduces burden for a protocol composer [4].
Protocol frameworks such as Appia and Eva extend the event-based approach with channels.
However, in our opinion, this solution is not satisfactory since composition of complex protocol stacks becomes more difficult.
In this paper, we propose a new approach for building modular protocols, that is based on a service abstraction.
We compare this new approach with the common, event-based approach.
We show that protocol frameworks based on services have several advantages, e.g. allow for a fairly straightforward protocol composition, clear implementation, and better support of dynamic replacement of distributed protocols.
To validate our claims, we have implemented SAMOA--an experimental protocol framework that is purely based on the service-based approach to module composition and implementation.
The framework allowed us to compare the service - and event-based implementations of an adaptive group communication middleware.
The paper is organized as follows.
Section 2 defines general notions.
Section 3 presents the main characteristics of event-based frameworks, and features that are distinct for each framework.
Section 4 describes our new approach, which is based on service abstraction.
Section 5 discusses the advantages of a service-based protocol framework compared to an event-based protocol framework.
The description of our experimental implementation is presented in Section 6.
Finally, we conclude in Section 7.
2.
PROTOCOL FRAMEWORKS
In this section, we describe notions that are common to all protocol frameworks.
Protocols and Protocol Modules.
A protocol is a distributed algorithm that solves a specific problem in a distributed system, e.g. a TCP protocol solves the reliable channel problem.
A protocol is implemented as a set of identical protocol modules located on different machines.
Protocol Stacks.
A stack is a set of protocol modules (of different protocols) that are located on the same machine.
Note that, despite its name, a stack is not strictly layered,
i.e. a protocol module can interact with all other protocol modules in the same stack, not only with the protocol modules directly above and below.
In the remainder of this paper, we use the terms machine and stack interchangeably.
Network
Figure 1: Example of a protocol stack
In Figure 1, we show an example protocol stack.
We represent protocol modules by capital letters indexed with a natural number, e.g. P1, Q1, R1 and S1.
We write Pi to denote the protocol module of a protocol P in stack i.
We use this notation throughout the paper.
Modules are represented as white boxes.
Arrows show module interactions.
For instance, protocol module P1 interacts with the protocol module Q1 and conversely (See Fig. 1).
Protocol Module Interactions.
Below, we define the different kinds of interaction between protocol modules.
• Requests are issued by protocol modules.
A request by a protocol module Pi is an asynchronous call by Pi of another protocol module.
• Replies are the results of a request.
A single request can generate several replies.
Only protocol modules belonging to the same protocol as the module that has issued the request are concerned by the corresponding replies.
For example, a request by Pi generates replies that concern only protocol modules Pj.
• Notifications can be used by a protocol module to inform (possibly many) protocol modules in the same stack about the occurrence of a specific event.
Notifications may also be the results of a request.
3.
EVENT-BASED PROTOCOL FRAMEWORK DESIGN
Most existing protocol frameworks are event-based.
Examples are Cactus [5, 2], Appia [1, 16] and Ensemble [12, 17].
In this section, we define the notion of an event in protocol frameworks.
We also explain how protocol modules are structured in event-based frameworks.
Events.
An event is a special object for indirect communication between protocol modules in the same stack.
Events may transport some information, e.g. a network message or some other data.
With events, the communication is indirect, i.e. a protocol module that triggers an event is not aware of the module (s) that handle the event.
Events enable one-to-many communication within a protocol stack.
Triggering an event can be done either synchronously or asynchronously.
In the former case, the thread that triggers an event e is blocked until all protocol modules that handle e have terminated handling of event e.
In the latter case, the thread that triggers the event is not blocked.
Protocol Modules.
In event-based protocol frameworks, a protocol module consists of a set of handlers.
Each handler is dedicated to handling of a specific event.
Handlers of the same protocol module may share data.
Handlers can be dynamically bound to events.
Handlers can also be unbound dynamically.
Upon triggering some event e, all handlers bound to e are executed.
If no handler is bound, the behavior is usually unspecified.
Figure 2: Example of an event-based protocol stack
In Figure 2, we show an example of an event-based stack.
Events are represented by small letters, e.g. e, f,...The fact that a protocol module can trigger an event is represented by an arrow starting from the module.
A white trapezoid inside a module box represents a handler defined by the protocol module.
To mark that some handler is bound to event e, we use an arrow pointing to the handler (the label on the arrow represents the event e).
For example, the protocol module P1 triggers event e and handles event f (see Fig. 2).
Note that the network is represented as a special protocol module that handles the send event (to send a message to another machine) and triggers the deliver event (upon receipt of a message from another machine).
Specific Features.
Some protocol frameworks have unique features.
Below, we present the features that influence composition and implementation of protocol modules.
In Cactus [5, 2], the programmer can give a priority number to a handler upon binding it to an event.
When an event is triggered, all handlers are executed following the order of priority.
A handler h is also able to cancel the execution of an event trigger: all handlers that should be executed after h according to the priority are not executed.
Appia [1, 16] and Eva [3] introduce the notion of channels.
Channels allow to build routes of events in protocol stacks.
Each protocol module has to subscribe to one or many channels.
All events are triggered by specifying a channel they belong to.
When a protocol module triggers an event e specifying channel c, all handlers bound to e that are part of a protocol that subscribes to c are executed (in the order prescribed by the definition of channel c).
4.
SERVICE-BASED PROTOCOL FRAMEWORK
In this section, we describe our new approach for implementing and composing protocols that is based on services.
We show in Section 5 the advantages of service-based protocol frameworks over event-based protocol frameworks.
Service Interface.
In our service-based framework, protocol modules in the same stack communicate through objects called service interfaces.
Requests, replies and notifications are all issued to service interfaces.
Protocol Modules.
A protocol module is a set of executers, listeners and interceptors.
Executers handle requests.
An executer can be dynamically bound to a service interface.
It can be later unbound.
A request issued to a service interface si leads to the execution of the executer bound to si.
If no executer is bound to si, the request is delayed until some executer is bound to si.
Contrary to events, at most one executer at any time can be bound to a service interface on every machine.
Listeners handle replies and notifications.
A listener can be dynamically bound and unbound to/from a service interface si.
A notification issued to a service interface si is handled by all listeners bound to si in the local stack.
A reply issued to a service interface is handled by one single listener.
To ensure that one single listener handles a reply, a module Pi has to identify, each time it issues a request, the listener to handle the possible reply.
If the request and the reply occur respectively, in stack i and in stack j, the service interface si on i communicates to the service interface si' on j the listener that must handle the reply.
If the listener that must handle the reply does not exist, the reply is delayed until the listener is created.
Network
Figure 3: Example of a service-based protocol stack
In Figure 3, we show an example of a service-based stack.
We denote a service interface by a small letter (e.g. t, u and nt) in a hexagonal box.
The fact that a module Pi can generate a request to a service interface si is represented by a dashed black arrow going from Pi to si.
Similarly, a dashed white arrow going from module Pi to service interface si represents the fact that Pi can generate a reply or a notification to si.
We represent executers with white boxes inside protocol modules and listeners with white boxes with a gray border.
A connecting line between a service interface si and an executer e (resp.
a listener l) shows that e (resp.
l) is bound to si.
In Figure 3, module Q1 contains an executer bound to service interface t and a listener bound to service interface u. Module Q1 can generate replies and notifications to service interface t and requests to service interface u. Note that the service interface nt allows to access the network.
Figure 4: Execution of protocol interactions with interceptors
An interceptor plays a special rˆole.
Similarly to executers, interceptors can be dynamically bound or unbound to a service interface.
They are activated each time a request, a reply or a notification is issued to the service interface they are bound to.
This is illustrated in Figure 4.
In the right part of the figure, the interceptor of the protocol module T1 is represented by a rounded box.
The interceptor is bound to service interface t.
The left part of the figure shows that an interceptor can be seen as an executer plus a listener.
When P1 issues a request req to the service interface t, the executer-interceptor of T1 is executed.
Then, module T1 may forward a request req' to the service interface t, where we can have req = 6 req' 1.
When module Q1 issues a reply or a notification, a similar mechanism is used, except that this time the listener-interceptor of T1 is executed.
Note that a protocol module Ti, that has an interceptor bound to a service interface, is able to modify requests, replies and notifications.
Upon requests, if several interceptors are bound to the same service interface, they are executed in the order of binding.
Upon replies and notifications, the order is reversed.
5.
ADVANTAGES OF SERVICE-BASED PROTOCOL FRAMEWORK DESIGN
We show in this section the advantages of service-based protocol frameworks over event-based protocol frameworks.
We structure our discussion in three parts.
Firstly, we present how protocol interactions are modeled in each of the protocol frameworks.
Then, we discuss the composition of protocol modules in each of these frameworks.
Finally, we present the problem of dynamic protocol replacement and the advantages of service interfaces in order to implement it.
The discussion is summarized in Table 1.
5.1 Protocol Module Interactions
A natural model of protocol interactions (as presented in Section 2) facilitates the implementation of protocol modules.
For each protocol interaction, we show how it is modeled in both frameworks.
We also explain that an inadequate model may lead to problems.
Requests.
In service-based frameworks, a request is generated to a service interface.
Each request is handled by at most one executer, since we allow only one executer to be bound to a service interface at any time.
On the other hand, in event-based frameworks, a protocol module emulates a request by triggering an event.
There is no guarantee
that this event is bound to only one handler, which may lead to programming errors.
Replies.
When a protocol module generates a reply in a service-based framework, only the correct listener (identified at the time the corresponding request was issued) is executed.
This ensures that a request issued by some protocol module Qi, leads to replies handled by protocol modules Qj (i.e. protocol modules of the same protocol).
This is not the case in event-based frameworks, as we now show.
Consider protocol module Q, in Figure 2 that triggers event g to emulate a request.
Module S, handles the request.
When modules Si triggers event h to emulate a reply (remember that a reply can occur in many stacks), both modules Qi and Ri will handle the reply (they both contain a handler bound to h).
This behavior is not correct: only protocol modules Qi should handle the reply.
Moreover, as modules Ri are not necessarily implemented to interact with modules Qi, this behavior may lead to errors.
Solutions to solve this problem exist.
However, they introduce an unnecessary burden on the protocol programmers and the stack composer.
For instance, channels allow to route events to ensure that modules handle only events concerning them.
However, the protocol programmer must take channels into account when implementing protocols.
Moreover, the composition of complex stacks becomes more difficult due to the fact that the composer has to create many channels to ensure that modules handle events correctly.
An addition of special protocol modules (named connectors) for routing events is also not satisfactory, since it requires additional work from the composer and introduces overhead.
Notifications.
Contrary to requests and replies, notifications are well modeled in event-based frameworks.
The reason is that notifications correspond to the one-to-many communication scheme provided by events.
In service-based frameworks, notifications are also well modeled.
When a module generates a notification to a service interface si, all listeners bound to s are executed.
Note that in this case, service interfaces provide the same pattern of communication as events.
5.2 Protocol Module Composition
Replies (and sometimes notifications) are the results of a request.
Thus, there is a semantic link between them.
The composer of protocol modules must preserve this link in order to compose correct stacks.
We explain now that service based frameworks provide a mechanism to preserve this link, while in event-based frameworks, the lack of such mechanism leads to error-prone composition.
In service-based frameworks, requests, replies and notifications are issued to a service interface.
Thus, a service interface introduces a link between these interactions.
To compose a correct stack, the composer has to bound a listener to service interface si for each module that issues a request to si.
The same must be done for one executer that is part of a module that issues replies or notifications.
Applying this simple methodology ensures that every request issued to a service interface si eventually results in several replies or notifications issued to the same service interface si.
In event-based frameworks, all protocol interactions are issued through different events: there is no explicit link between an event triggered upon requests and an event triggered upon the corresponding replies.
Thus, the composer of a protocol stack must know the meaning of each event in order to preserve the semantic link between replies (and notifications) and requests.
Moreover, nothing prevents from binding a handler that should handle a request to an event used to issue a reply.
Note that these problems can be partially solved by typing events and handlers.
However, it does not prevent from errors if there are several instances of the same event type.
Note that protocol composition is clearer in the protocol frameworks that are based on services, rather than on events.
The reason is that several events that are used to model different protocol interactions can be modeled by a single service interface.
5.3 Dynamic Replacement of Protocols
Dynamic replacement of protocols consists in switching on-the-fly between protocols that solve the same problem.
Replacement of a protocol P by a new protocol newP means that a protocol module Pi is replaced by newPi in every stack i.
This replacement is problematic since the local replacements (within stacks) must be synchronized in order to guarantee protocol correctness [21, 18].
Figure 5: Dynamic replacement of protocol P
For the synchronization algorithms to work, module interactions are intercepted in order to detect a time when Pi should be replaced by newPi.
(Other solutions, e.g. in [11], are more complex.)
In Fig. 5, we show how this interception can be implemented in protocol frameworks that are based on services (in the left part of the figure) and events (in the right part of the figure).
The two-sided arrows point to the protocol modules P, and newP, that are switched.
It can be seen that the approach that uses the Service Interface mechanism has advantages.
The intercepting module Repl-P, has an interceptor bound to service interface t that intercepts every request handled by modules P, and all replies and notifications issued by P,.
The code of the module P, can therefore remain unchanged.
In event-based frameworks, the solution is to add an intermediate module Repl-P, that intercepts the requests issued to P, and also the replies and notifications issued by P,.
Although this ad-hoc solution may seem similar to the servicebased approach, there is an important difference.
The eventbased solution requires to slightly modify the module P, since instead of handling event g and triggering event h, P, must now handle different events g' and h' (see Fig. 5).
6.
IMPLEMENTATION
We have implemented an experimental service-based protocol framework (called SAMOA) [7].
Our implementation is light-weight: it consists of approximately 1200 lines of code in Java 1.5 (with generics).
In this section, we describe the main two classes of our implementation: Service (encoding the Service Interface) and
Table 1: Service-based vs. event-based
Protocol (encoding protocol modules).
Finally, we present an example protocol stack that we have implemented to validate the service-based approach.
The Service Class.
A Service object is characterized by the arguments of requests and the arguments of responses.
A response is either a reply or a notification.
A special argument, called message, determines the kind of interactions modeled by the response.
A message represents a piece of information sent over the network.
When a protocol module issues a request, it can give a message as an argument.
The message can specify the listener that must handle the reply.
When a protocol module issues a response to a service interface, a reply is issued if one of the arguments of the response is a message specifying a listener.
Otherwise, a notification is issued.
Executers, listeners and interceptors are encoded as innerclasses of the Service class.
This allows to provide type-safe protocol interactions.
For instance, executers can only be bound to the Service object, they belong to.
Thus, the parameters passed to requests (that are verified statically) always correspond to the parameters accepted by the corresponding executers.
The type of a Service object is determined by the type of the arguments of requests and responses.
A Service object t is compatible with another Service object s if the type of the arguments of requests (and responses) of t is a subtype of the arguments of requests (and responses) of s.
In practice, if a protocol module Pi can issue a request to a protocol UDP, then it may also issue a request to TCP (compatible with UDP) due to the subtyping relation on parameters of communicating modules.
The Protocol Class.
A Protocol object consists of three sets of components, one set for each component type (a listener, an executer, and an interceptor).
Protocol objects are characterized by names to retrieve them easily.
Moreover, we have added some features to bind and unbind all executers or interceptors to/from the corresponding Service objects.
Protocol objects can be loaded to a stack dynamically.
All these features made it easy to implement dynamic replacement of network protocols.
Protocol Stack Implementation.
To validate our ideas, we have developed an Adaptive Group Communication (AGC) middleware, adopting both the service - and the event-based approaches.
Fig. 6 shows the corresponding stacks of the AGC middleware.
Both stacks allow the Consensus and Atomic Broadcast protocols to be dynamically updated.
The architecture of our middleware, shown in Fig. 6, builds on the group communication stack described in [15].
The UDP and RP2P modules provide respectively, unreliable and reliable point-to-point transport.
The FD module implements a failure detector; we assume that it ensures the
Figure 6: Adaptive Group Communication Middleware: service-based (left) vs. event-based (right)
properties of the ✸ S failure detector [9].
The CT module provides a distributed consensus service using the ChandraToueg algorithm [10].
The ABc.
module implements atomic broadcast--a group communication primitive that delivers messages to all processes in the same order.
The GM module provides a group membership service that maintains consistent membership data among group members (see [19] for details).
The Repl ABc.
and the Repl CT modules implement the replacement algorithms [18] for, respectively, the ABc.
and the CT protocol modules.
Note that each arrow in the event-based architecture represents an event.
We do not name events in the figure for readability.
The left stack in Figure 6 shows the implementation of AGC with our service-based framework.
The right stack shows the same implementation with an event-based framework.
Performance Evaluation.
To evaluate the overhead of service interfaces, we compared performance of the serviceand event-based implementations of the AGC middleware.
The latter implementation of AGC uses the Cactus protocol framework [5, 2].
In our experiment, we compared the average latency of Atomic Broadcast (ABcast), which is defined as follows.
Consider a message m sent using ABcast.
We denote by ti (m) the time between the moment of sending m and the moment of delivering m on a machine (stack) i.
We define the average latency of m as the average of ti (m) for all machines (stacks) i within a group of stacks.
Performance tests have been made using a cluster of PCs running Red Hat Linux 7.2, where each PC has a Pentium III 766 MHz processor and 128MB of RAM.
All PCs are interconnected by a 100 Base-TX duplex Ethernet hub.
Our experiment has involved 7 machines (stacks) that ABcast messages of 4Mb under a constant load, where a load is a number of messages per second.
In Figure 7, we show the results of our experiment for different loads.
Latencies are shown on the vertical axis, while message loads are shown on the horizontal axis.
The solid line shows the results obtained with our service-based framework.
The dashed line shows the results obtained with the Cactus framework.
The Network Network
Figure 7: Comparison between our service-based framework and Cactus
overhead of the service-based framework is approximately 10%.
This can be explained as follows.
Firstly, the servicebased framework provides a higher level abstraction, which has a small cost.
Secondly, the AGC middleware was initially implemented and optimized for the event-based Cactus framework.
However, it is possible to optimize the AGC middleware for the service-based framework.
7.
CONCLUSION
In the paper, we proposed a new approach to the protocol composition that is based on the notion of Service Interface, instead of events.
We believe that the service-based framework has several advantages over event-based frameworks.
It allows us to: (1) model accurately protocol interactions, (2) reduce the risk of errors during the composition phase, and (3) simply implement dynamic protocol updates.
A prototype implementation allowed us to validate our ideas.

servic interfac
protocol framework
modul
modular
network
distribut algorithm
distribut system
commun
event-base framework
stack
request
repli
dynam protocol replac