# G2CL: A Generic Group Communication Layer for Clustered Applications

Leandro Sales, Henrique Teófilo, Nabor C. Mendonça

► To cite this version:

Leandro Sales, Henrique Teófilo, Nabor C. Mendonça. G2CL: A Generic Group Communication Layer for Clustered Applications. 10th IFIP WG 6.1 International Conference on Distributed Applications and Interoperable Systems (DAIS) / Held as part of International Federated Conference on Distributed Computing Techniques (DisCoTec), Jun 2010, Amsterdam, Netherlands. pp.169-182, 10.1007/978-3-642-13645-0_13 . hal-01061087

HAL Id: hal-01061087

https://inria.hal.science/hal-01061087

Submitted on 5 Sep 2014

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# G2CL: A Generic Group Communication Layer for Clustered Applications

Leandro Sales<fnref n="1" />, Henrique Teófilo<fnref n="2" />, and Nabor C. Mendonça<fnref n="2" />

**Abstract.** Generic group communication frameworks offer several benefits to developers of clustered applications, including better software modularity and greater flexibility in selecting a particular group communication system. However, current generic frameworks only support a very limited set of group communication primitives, which has hampered their adoption by many “real-world” clustered applications that require higher-level group communication services, such as state transfer, distributed data structures and replicated method invocation. This paper describes the design, implementation and initial evaluation of G2CL, a Generic Group Communication Layer that offers a set of commonly used high-level group communication services implemented on top of an existing generic framework. Compared to current group communication solutions, G2CL offers two main contributions: (i) its services can be configured to run over any group communication system supported by the underlying generic framework; and (ii) it implements the same service API used by JGroups, a popular group communication toolkit, which may reduce its learning curve and make the task of migrating to G2CL particularly attractive for JGroups users.

## 1 Introduction

Group communication, i.e., the ability to reliably transmit messages amongst a group of processes, plays an important role in the design of dependable and adaptable distributed systems [8]. This form of communication has been particularly valuable in clustered environments, where classical group communication applications include replication, load balancing, resources management and monitoring, and highly available services [7].

A group communication system (GCS) is a type of middleware that implements a set of reusable group communication services that can be useful in multiple application domains. Some of the most popular GCSs currently in use are JGroups [4], Spread [2] and Appia [20], each providing its own set of group communication primitives and protocols. Choosing an appropriate GCS for a

<footnote n="1">¹ Dipartimento di Elettronica e Informazione, Politecnico di Milano, Via Ponzio, 34/5 – 20133 Milano, Italy, pinto@elet.polimi.it</footnote>

<footnote n="2">² Mestrado em Informática Aplicada, Universidade de Fortaleza (UNIFOR), Av. Washington Soares, 1321 – 60811-905 Fortaleza – CE, Brazil, henriquetft@gmail.com, nabor@unifor.br</footnote>
given distributed application is an important design decision that can be made
difficult by the fact that those systems tend to vary widely not only in terms of
the communication abstractions they implement, but also in terms of the deliv-
ery semantics and quality-of-service (QoS) guarantees they provide [7]. Another
difficulty is that, once a developer commits to a particular GCS, her application
code becomes tightly coupled to that system’s API. Such level of coupling is
undesirable for two main reasons: (i) it requires changing the application code
every time the target API evolves; and (ii) it makes it extremely hard to mi-
grate the application to a different GCS (with a different API), thus preventing
developers from easily benefiting from a new (possibly more effective) GCS in
the future.

An interesting way for developers of distributed applications to avoid cou-
pling their application code to the services provided by a specific middleware
solution is to use a generic middleware API. Typically, such generic APIs are
implemented by means of a plug-in mechanism which allows application devel-
opers to select a particular concrete middleware system at configuration time,
without the need to change their application code. This approach has been suc-
cessfully used in a number of distributed software domains, including structured
peer-to-peer communication [9], publish-subscribe systems [22] and grid appli-
cations [21].

In terms of group communication, there have been some attempts to pro-
vide a common API for different GCSs, such as in Hedera [13], jGCS [6] and
Shoal [26]. However, all those systems only implement basic operations for re-
liable message transmission and group management. The problem, in this case,
is that some mature GCSs, such as JGroups, also offer a number of higher-
level group-related services, for instance, object state transfer, distributed data
structures and transparent invocation of replicated objects. As a consequence,
many “real-world” distributed applications that rely on those high-level services
cannot benefit from existing generic group communication APIs.

In this paper, we describe the design, implementation and initial evaluation
of G2CL, a Generic Group Communication software Layer that implements a
set of commonly used high-level group communication services, similarly to those
already provided by JGroups. In contrast to JGroups, though, all services pro-
vided by G2CL are implemented on top of an existing generic API, which allows
them to be easily reconfigured to run over any GCS supported by the underly-
ing plug-in mechanism. To demonstrate the power of G2CL we have successfully
used it to replace JGroups as the generic group communication solution for the
JOnAS JEE application server [17]. The migration from JGroups to G2CL in
the JOnAS source code has been done with relatively little programming effort,
as we will describe later in the paper, and has allowed us to evaluate the impact
of using different G2CL configurations on the performance of JOnAS under a
variety of load conditions. These results build our confidence that G2CL can
be a valuable addition to the set of programming tools currently available for
developers of distributed applications.
The rest of the paper is organized as follows. Section 2 gives a brief overview of two technologies that have greatly influenced our work on G2CL, namely JGroups and jGCS. Section 3 describes the main design decisions and implementation strategies used in the development of G2CL. Section 4 reports on our initial evaluation of G2CL using JOnAS as a case study. Section 5 further discusses our results and highlights the merits and limitations of our work. Finally, Section 6 concludes the paper and outlines our future research agenda.

# 2 Related Technologies

## 2.1 JGroups

JGroups [4] was one of the first group communication toolkits written entirely in Java. It provides a simple API for accessing its basic group communication services, whose main component is the *Channel* interface. This interface is used to send/receive messages asynchronously to/from a group of processes, and to monitor group changes by means of the *Observer* design pattern [12]. Currently, JGroups offers a single implementation of the *Channel* interface, called *JChannel*[^3].

The *Channel* interface hides the actual protocol stack used by JGroups for message transmission. However, JGroups allows developers to configure their own protocol stack, by combining different protocols for message transmission (for instance, TCP or UDP over IP Multicast), data fragmentation, reliability, security, failure detection, membership control, etc. This can be done via an external XML file, whose properties are loaded by JGroups at initialization time, thus avoiding the need to change the application code directly.

On top of the basic services provided by the *Channel* interface, JGroups implements another set of higher-level services, called *building blocks* [16], which offer more sophisticated group communication abstractions for application developers. These include services such as MessageDispatcher, which implements primitives for synchronous message transmission; RPCDispatcher, which implements a remote invocation mechanism for replicated objects on top of the MessageDispatcher service; and ReplicatedHashMap, which implements a distributed version of the *HashMap* class of Java on top of the RPCDispatcher service.

Due to its great flexibility in defining customized protocol stacks, and also to its rich set of building blocks, JGroups has been a popular choice amongst clustered application developers, having recently been incorporated as part of the JBoss project [15].

## 2.2 jGCS

The *Group Communication Service for Java* (jGCS) [6] is a generic group communication framework that aims at providing a common Java API to several

<footnote>[^3]: In our work, we have used JGroups version 2.6.10, released on April 28, 2009. JGroups is available at http://www.jgroups.org.</footnote>
existing GCSs. Its ultimate goal is to facilitate reuse of the different services im-
plemented by those systems without requiring substantial changes in the source
code of the target application.

The jGCS architecture relies on a plug-in mechanism based on the *Inversion of Control* (IoC) design pattern [10]. This mechanism is used by jGCS to decouple its service API from the underlying service implementation, thus allowing the same API to be reused across different GCSs. The actual service implementation (plug-in) used by jGCS can be defined at initialization time, via an external configuration file. The current version of jGCS offers plug-ins for several GCSs, including JGroups, Spread [2] and Appia [20].<fnref n="4" />

The jGCS API is divided into four complementary interfaces, namely *configuration interface*, *common interface*, *data interface*, and *control interface*. These are described in more details below.

*Configuration Interface.* This interface decouples the application code from implementation-dependent group communication concerns, such as group configuration and specification of message delivery guarantees. The actual GCS plug-in to be used is defined at configuration time, by means of an external configuration file. At execution time, the jGCS services are instantiated according to the specified configuration, using a dependency injection mechanism [10] or a service locator [1].

The main classes of this interface are *ProtocolFactory*, which implements the *Abstract Factory* design pattern [12] to allow the initialization of new protocol instances based on the underlying plug-in configuration; *GroupConfiguration*, which encapsulates group information (e.g., the group ID) necessary to open a new group session through which the application can exchange messages with other group members and monitor group membership changes; and *Service*, which encapsulates the specification of message delivery guarantees to be used during message transmission.

*Common Interface.* This interface contains common classes shared by all other interfaces. The main class of this interface is *Protocol*, whose instances are created by the *ProtocolFactory* class from the configuration interface. A *Protocol* object is used to create, for a given *GroupConfiguration* object, the objects responsible for message exchange and group membership management, of types *DataSession* and *ControlSession*, respectively, described next.

*Data Interface.* This interface contains classes responsible for sending and receiving group messages. The main classes of this interface are *DataSession*, which is used to send messages to a group and also to register *observers* [12] to handle messages received from that same group; *Message*, which encapsulates a message to be sent or received from a group and the address of the sender; and *MessageListener*, which must be implemented by all *observers* registered with a *DataSession*.

<footnote n="4">⁴ In our study, we have used jGCS version 0.6.1, released on October 29, 2007. jGCS is available at http://jgcs.sourceforge.net/.</footnote>
To avoid forcing any specific data format or serialization mechanism on the
application, the message body is stored as a *byte array* inside *Message*, with the
application being responsible for serializing the message before transmission and
deserializing it after receipt.

*Control Interface.* This interface contains classes responsible for group manage-
ment, from simple notifications of members joining or leaving a group to the
creation of new virtual group views. The main classes of this interface are *Con-
trolSession*, which provides methods for members to join or leave a group and
also to register *observers* to listen to notifications of membership changes (e.g.,
join, leave and failure of members); *ControlListener*, which must be implemented
by all *observers* registered with a *ControlSession*; *MembershipSession*, which is
an extension of class *ControlSession* used to obtain a list of members currently
connected to a group and also to register *observers* to listen to changes in group
views; and *MembershipListener*, which must be implemented by all *observers*
registered with a *MembershipSession*.

3 G2CL

G2CL is an extensible group communication software layer that sits on top of existing generic frameworks. Its main design goal is to offer a more sophisticated set of generic group communication services, similar to those provided by the building blocks of JGroups, but with all the benefits associated with the use of a loosely-coupled software architecture. To achieve this goal, we have taken some important design decisions, discussed below.

## 3.1 Main Design Decisions

**Choice of Generic Framework.** Our first design decision was concerned with selecting the generic framework to be used as the basis for the implementation of G2CL. Of the three generic frameworks currently available, i.e., Hedera [13], jGCS [6] and Shoal [26], only Hedera and jGCS were considered mature enough for our purposes, with both providing plug-ins for several existing GCSs. Shoal, on the other hand, only provides support for a single GCS (namely, JXTA [18]) and thus was discarded as a possible generic framework candidate.

The choice between Hedera and jGCS was based on several factors, including
an analysis of their design features and performance overhead. In the end, we
chose to use jGCS because of its well-designed API, which has been implemented
following well-known object-oriented design principles and patterns [6], and the
fact that it offers a much lower performance overhead compared to the overhead
imposed by Hedera, particularly for small messages [24,25].

**Service Implementation Model.** Another important design decision was concerned with defining an appropriate implementation model for G2CL. Given the rich set of group communication building blocks offered by JGroups, and its
popularity amongst distributed application developers, we have decided to im-
plement the G2CL services following, whenever possible, the same building block
API (including class names and method signatures) used by JGroups. This de-
cision has the potential to facilitate the task of migrating an existing clustered
application based on JGroups to G2CL, since both systems implement similar
APIs. Another benefit is that G2CL users could greatly reduce their learning
curve by leveraging on JGroups' extensive API documentation and code base.

**jGCS Extensions.** During the design of G2CL we have identified the need to
make some minor extensions to the classes and interfaces originally provided by
jGCS. These extensions are described below.

As it is typical with other communication abstractions that encapsulate
lower-level services, to implement some of the G2CL services we needed a way
to add service-specific headers to application messages in a manner that is sep-
arate from their body. Such headers would be used to store control information
relevant to the implementation of some services, but which could not be exposed
to the application. Since this facility is not readily supported by the *Message*
class currently provided by jGCS, we had to define a new message class, called
*G2CLMessage*.

In order to maintain compatibility with the *DataSession* class of jGCS,
*G2CLMessage* implements jGCS’s *Message* interface. This allows *G2CLMessage*
objects to be transmitted as any other message using any jGCS plug-in.

Another extension made to jGCS was the implementation of a new *DataSession* class, called * MarshalDataSession ,* which works like an adapter [12] between the G2CL services and the original *DataSession* used by the jGCS plug-ins. The main responsibility of this new class is to intercept all message transmission calls made to the plug-in by the application and then execute the necessary trans- formations to convert between a message of type * G2CLMessage * and another message of type * Message . In this way, all G2CL services must rely only on * Mar- shalDataSession * for message transmission (instead of the original * DataSession * class of jGCS).

3.2 Implemented Services

The initial set of group communication services implemented as part of G2CL
was selected based on an informal analysis of the JGroups services that are most
commonly used in practice. The selected services were classified into two groups,
named *high-level services* and *service decorators*, described below.

**High-level Services.** These services encapsulate a * MarshalDataSession * instance by hiding its basic message transmission functionality, so as to provide application developers with a more sophisticated group communication API. Four services were initially implemented as part of this group: * MessageDispatcher , * RpcDispatcher , * ReplicatedHashMap * and * StateTransferDataSession . Those four services are briefly described below.
**MessageDispatcher** Provides a way to send synchronous messages to the group with request-response correlation. Sending synchronous message to the group can cause ambiguity in regards to when the execution should resume. Hence, the sender should choose between different policies that specify how many members should receive the message before considering the message as sent.

**Rpcreceiver** Provides a way to make remote method invocation in the members of the group. When creating his own `Rpcreceiver` instance, each member needs to specify the object in which the received method invocations should be made. As method invocations are synchronous, to avoid ambiguity, as in `MessageDispatcher`, the invoker needs to choose between different policies to specify when the method should return.

**StateTransferDataSession** Provides a `DataSession` with a state transfer mechanism implemented based on the JGroups State Transfer service [5]. It should be used when the application needs to maintain a replicated state amongst all group members.

**ReplicatedHashMap** Implements a `Map` object replicated across all members of the group. Any change to the map (via invocation of `clear()`, `put()`, `remove()`, etc.) will transparently be propagated to all replicas in the group. Invocations of read-only methods always access the local replica.

Due to space limitations, and because those services provide the same set of functionalities provided by their corresponding services in JGroups, with a similar API, we will omit the details of their implementation from the paper. For a more detailed account of those services, the interested reader is referred to [11].

**Service Decorators.** Services of this group add extra functionalities (such as message fragmentation and encryption) to the basic message transmission service provided by the * MarshalDataSession * class. As the group name implies, these services are based on the * Decorator * design pattern [12]. Their implementation keeps the same interface provided by * MarshalDataSession * , so that their use is completely transparent to the application.

Currently, G2CL provides four service decorators, namely * FragDataSession *, * BundleDataSession *, * CompressDataSession * and * CryptoDataSession *. These ser- vices provide mechanisms for message fragmentation, message bundle, message compression and message encryption, respectively.

Each service decorator can be used either in isolation, or combined with other service decorators, forming a *chain of responsibility* [12] where different decora- tors can be added or removed from the chain without affecting the application code.

To facilitate the use and configuration of service decorators, G2CL provides a * MarshalDataSessionFactory * class whose main responsibility is to create a new * MarshalDataSession * instance. If necessary, the * MarshalDataSessionFactory * can also instantiate a chain of decorators for the new * MarshalDataSession * object.
The creation of both the * MarshalDataSession * instance and its chain of dec-
orators can be configured by the user in a manner that is independent of the
application code, using a dependency injection mechanism or a service locator.

# 4 Evaluation

To assess the migration effort and potential performance impact associated with
the use of G2CL in a real clustered application, we have conducted a case study
involving the JOnAS Java EE application server [17]. The reason for selecting
JOnAS as our target application is two-fold: (i) it is a mature clustered tech-
nology of non-trivial size (in the order of 230.000 lines of Java code); and (ii) it
makes intensive use of a number of group communication services and building
blocks provided by JGroups, which have similar services already implemented
as part of G2CL.

## 4.1 JOnAS Overview

The *Java Open Application Server* (JOnAS) is an open source implementation
of the Java EE 5 specification [27].<fnref n="5" /> JOnAS supports the creation of reliable EJB
applications by providing a high-availability (HA) service based on a cluster of
JOnAS instances. When a client application requests the creation of a *Stateful
Session Bean* (SFSB) component, one of the servers in the cluster is chosen
to respond to that client’s invocations until the client requests the removal of
that SFSB. Before sending a response to the client, the server propagates any
change in the state of the SFSB to the other servers in the cluster, which act as
backup servers for that component. If the server initially allocated to a replicated
component fails, the state of the SFSB can be recovered by one of its backup
servers, which will start handling future invocations for that component on behalf
of the failed server.

To implement its HA service JOnAS relies on a RMI-like replication protocol
called *Clustered Method Invocation* (CMI), which is specifically tailored for trans-
parently invoking replicated objects. The CMI protocol uses several high-level
group communication services provided by JGroups to implement a number of
features, including a distributed version of a JNDI-based resource registry, and
a state propagation mechanism. More specifically, the distributed registry uses
the *RPCDispatcher* and *StateTransfer* services of JGroups to guarantee that
any changes made to registry by one of the servers are reliably propagated to
the other servers (for instance, when a new object is created). The state prop-
agation mechanism, in turn, uses the *MessageDispatcher* service of JGroups to
guarantee that, whenever the server responsible for a replicated object fails, at
least one of the remaining servers in the cluster will have all the necessary
information to continue responding to any ongoing or future client request on
behalf of the failed server.

<footnote n="5">⁵ In our work, we have used JOnAS version 5.1.0-M5. JOnAS is available at
http://jonas.ow2.org.</footnote>
In the following we describe how we have replaced, in the JOnAS source code, all JGroups services used in the implementation of the CMI protocol with the corresponding generic services provided by G2CL.

## 4.2 Migration Process

Our migration process was concentrated on two JOnAS classes, namely *Syn-chronyzedDistributedTree* and *JGMessageManager*. These are the main classes involved in the implementation of the distributed registry and the replication mechanism of CMI, respectively.

In both classes our migration strategy consisted, essentially, of changing all lines of code (and, when necessary, their associated configuration files) responsible for initializing the target JGroups services (i.e., *RPCDispatcher*, *StateTrans-fer* and *MessageDispatcher*), in order to replace them with the necessary code to initialize the corresponding services of G2CL.

One notable exception was the need to implement a new message serialization mechanism for JOnAS. This was required because the original version of JOnAS uses the serialization mechanism provided by JGroups, while jGCS (and, consequently, G2CL) leaves the serialization process to be implemented by the application.

Finally, we also had to change the way JOnAS handles the identification of group members. In the original version of JOnAS, group members are identified by the *Address* class of JGroups. In the new version, based on G2CL, this class was replaced by the *SocketAddress* class, which is the class used to identify group members in jGCS.

It is interesting to note that, even though the JGroups services we have replaced are actually *used* in many other parts of the JOnAS source code, we did not have to change any of those parts. This was due to our decision to keep the same JGroups API when implementing the corresponding services in G2CL.

Table 1 quantifies our migration effort in terms of the number of JOnAS packages, classes and lines of code (LoC) that had to be modified as part of our G2CL migration strategy. From that table we can see that most of the changes were performed in the CMI module, where about 2.5% of its packages, 2.8% of its classes and 11% of its lines of code had to be modified. These numbers reflect the fact that CMI makes intensive use of JGrous in its implementation, as we have explained above. Even though many of the changes made to the CMI module were certainly non-trivial, we can still see these numbers in a positive light if we consider that nearly 98% of the packages and classes of that module (comprising about 90% of its lines of code) were left unchanged after the migration. The percentage of changes in the other modules was much smaller, as expected, varying between 0.06 and 0.8%. Overall, we only had to change about 1% of the total of lines in the JOnAS source code.

The above numbers are indicative that the programming effort required by the G2CL migration process was relatively low compared to the full size of the JOnAS source code. They also reflect the fact that group communication,
<table><caption>Table 1. JOnAS migration numbers</caption><thead><tr><th rowspan="2">Module</th><th colspan="2"># Packages</th><th colspan="2"># Classes</th><th colspan="2"># LoC</th></tr><tr><th>Total</th><th>Changed (%)</th><th>Total</th><th>Changed (%)</th><th>Total</th><th>Changed (%)</th></tr></thead><tbody><tr><td>CMI</td><td>80</td><td>2 (2,50%)</td><td>216</td><td>6 (2,78%)</td><td>18.691</td><td>2.106 (11,28%)</td></tr><tr><td>OW2-UTIL</td><td>235</td><td>1 (0,42%)</td><td>596</td><td>5 (0,84%)</td><td>33.538</td><td>270 (0,80%)</td></tr><tr><td>JOnAS</td><td>396</td><td>1 (0,25%)</td><td>2133</td><td>1 (0,04%)</td><td>180.030</td><td>111 (0,06%)</td></tr><tr><td>All</td><td>711</td><td>4 (0,56%)</td><td>2945</td><td>12 (0,41%)</td><td>232.259</td><td>2.487 (1,01%)</td></tr></tbody></table>

although crucial to the provisioning of some important services of JOnAS, is only used scarcely in its implementation.

## 4.3 Performance Impact Analysis

Despite the clear software engineering benefits that can be associated with the use of generic APIs, one cannot neglect the inevitable performance impact that those systems may impose on the services they generalize. With this concern in mind, we have analysed the potential overhead caused by G2CL on the performance of JOnAS. Our analysis compared the performance of the original version of JOnAS, based on JGroups, against that of the new version, based on G2CL, using three different jGCS plug-in configurations.

**Method.** Our analysis was carried out in a local cluster environment, which was configured in a manner to emulate a typical JEE clustering scenario [19]. This environment was composed of nine PCs connected through a dedicated 10/100 Mbps Fast Ethernet switch. Each PC had the following configuration: Intel Core 2 Duo processor; 2 GB RAM (DDR2); and Linux Debian (version 5.0) operating system.

Six PCs were used in the business layer, each one running a separate JOnAS instance with CMI and the HA service enabled, playing the role of replicated EJB containers. Two other PCs were used in the presentation layer, each one also running a separate JOnAS instance, but now playing the roles of both web containers and CMI clients. Finally, one PC was used to run the Apache server (version 2.2.11), which was responsible for balancing the load amongst the servers of the presentation layer.

To compare the performance of the different JOnAS versions, we have developed a simple EJB application with a single SFSB. This SFSB implements the basic functionalities of a shopping cart in an e-commerce application, offering operations to insert, update and remove items from the shopping cart. For persistence, we used the PostgreSQL relational database system (version 8.3) [23]. This EJB application was installed in all the six servers of the business layer, with its SFSB component being configured as a replicated CMI object.
We have also developed a simple web-based client application to continuously
invoke a series of operations provided by the replicated object (shopping cart)
at the business layer. Both the EJB application and the client application were
implemented in a way to create an execution scenario similar to the one used
by Lodi et al. in [19], where the authors have compared the performance of an
enhanced version of the JBoss application server [15].

We ran multiple sets of experiments, with each experiment involving a dif-
ferent version of JOnAS. In total, we analysed the performance of four JOnAS
versions: the original version, based on JGroups, and three variations of the new
version, based on G2CL, using the jGCS plug-ins for JGroups, Spread and Appia,
respectively. In all experiments we varied the number of clients from 50 to 100,
so as to observe the performance of the different versions of JOnAS under differ-
ent load conditions. To generate the client loads we used the ApacheBench(ab)
benchmarking tool (version 2.0) [3].

In terms of group communication features, we configured the three jGCS
plug-ins to provide the same set of guarantees that is provided by JGroups in
the original version of JOnAS. This was necessary to make sure that the new
version of JOnAS, based on G2CL, would behave, at least functionally, in a
similar fashion to its original version.

Finally, we used the *client response time* as our performance measure [14]. In our analysis, this measured as computed by calculating the average response time observed across all clients during the same experiment. To achieve a confidence interval of 95%, each experiment was executed at least 30 times, with extreme outliers being removed using the *boxplots* method [28].

**Results.** Figure 1 shows the average client response time observed for the four versions of JOnAS as a function of the number simultaneous client requests handled by the EJB application. As we can see, the different JOnAS versions are non-uniformly affected as the number of client requests grows. In addition, when we compare the original version of JOnAS, which uses JGroups directly, against the new version, based on G2CL configured with the JGroups plug-in, we note that their performances is very close, with a slight advantage to the former. This shows that the performance overhead imposed by G2CL on JOnAS is minimal (for 50 simultaneous requests, their performance differ by about 27% in favor of the original version, with that difference quickly falling below 5% as the number of simultaneous requests approaches the 70 mark).

We also observed that the new JOnAS version configured with the Appia
plug-in imposes a virtually constant performance loss (in the order of 25%)
when compared with its original version. When the new version is configured
with the Spread plug-in, the observed performance loss is even higher (up to
42% for 100 simultaneous requests).

These results suggest that the performance impact imposed by the use of
G2CL may not be determined a priori, as it is likely to be influenced by the
performance of the underlying jGCS plug-in. In this regard, we believe G2CL
can offer a real contribution towards more effective clustering solutions, since it
<figure><img src="image_2.png" /><figcaption>Fig. 1. Performance analysis results.</figcaption></figure>

liberates developers to experiment with new group communication mechanisms
without requiring a significant programming effort.

# 5 Discussion

In our work, we use the term *generic* to convey the notion of flexibility and portability. In this sense, we say that both jGCS and G2CL implement generic APIs, in that both can be easily configured to use different GCSs as their underlying group communication mechanism. Similarly, we say that JGroups implements a specific (non-generic) API, since its services are tightly-coupled to its own group communication primitives and protocols. In terms of group communication abstractions, G2CL adds little extra functionality beyond those already provided by jGroups. However, compared with JGroups, G2CL main advantage is its greater flexibility for configuring its underlying group communication mechanism, which can be any of the GCSs currently supported by jGCS. Therefore, by providing a flexible implementation, based on jGCS, for a number of commonly used JGroups services, G2CL combines the best of both systems.

With respect to its performance impact, our early experimental results from the JOnAS case study show that G2CL offers a noticeable yet non significant performance loss compared with the performance of JGroups when the latter is used in standalone mode, particularly under high load conditions. Nonetheless, we believe that the use of a generic group communication API can still pay-off in terms of improving application performance. As we have already shown elsewhere [24,25], Spread can outperform JGroups by a large margin under certain communication scenarios. This means that, for some distributed applications that use JGroups, the migration to G2CL using a Spread-based configuration
might actually result in a real performance gain. A further investigation of the conditions upon which migrating to G2CL might improve application performance is an interesting topic for future work.

An important limitation of our work thus far is that we have limited our evaluation to single performance metric (i.e., client response time). In this regard, we plan to further investigate the potential impact of using different G2CL configurations on other well-known performance metrics, such as server throughput and memory consumption.

# 6 Conclusion

In this paper, we have presented our work on G2CL, a generic software layer providing a rich set of high-level group communication services. Our early experience in using G2CL in the JOnAS application server as well as in other middleware technologies suggests that it can be effectively used as a generic group communication solution for existing clustered technologies, requiring a relatively modest migration effort and imposing a minimal performance overhead, particularly for those applications originally based on JGroups.

A natural line for future research is to improve G2CL with new group communication services and features. We are also conducting more case studies, involving open source clustered applications of varying sizes and domains and using new performance metrics, in order to better analyse the benefits and limitations of our approach.

G2CL is being developed as an open source project. Its source code and documentation are freely available at http://g2cl.googlecode.com.

# References

1. Alur, D., Malks, D., Crupi, J., Booch, G., Fowler, M.: Core J2EE Patterns: Best Practices and Design Strategies. Sun Microsystems, Inc., Mountain View, CA, USA, 2nd. edn. (2001)

2. Amir, Y., Danilov, C., Stanton, J.: A Low Latency, Loss Tolerant Architecture and Protocol for Wide Area Group Communication. In: Proceedings of the 2000 International Conference on Dependable Systems and Networks (FTCS-30, DCCA-8). pp. 327–336. IEEE CS Press, New York, NY, USA (2000)

3. Apache: Apache HTTP server benchmarking tool (1996), http://httpd.apache.org/docs/2.0/programs/ab.html

4. Ban, B.: Design and Implementation of a Reliable Group Communication Toolkit for Java. Tech. rep., Cornell University, Cornell University (1998)

5. Ban, B.: A Flexible API for State Transfer in the JavaGroups Toolkit (2007), unpublished manuscript

6. Carvalho, N., Pereira, J., Rodrigues, L.: Towards a Generic Group Communication Service. In: Proceedings of the 8th International Symposium on Distributed Objects and Applications (DOA’06). pp. 1485–1502. Springer, Montpellier, France (2006)

7. Chockler, G.V., Keidar, I., Vitenberg, R.: Group Communication Specifications: A Comprehensive Study. ACM Computing Surveys 33(4), 427–469 (2001)
8. Couloris, G., Dollimore, J., Kindberg, T.: Distributed Systems – Concepts and Design. Addison-Wesley, Boston, MA, USA, 4th edn. (2005)

9. Dabek, F., Zhao, B., Druschel, P., Kubiatowicz, J., Stoica, I.: Towards a Common API for Structured Peer-to-Peer Overlays. In: Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS’03). pp. 33–44. Springer, Berkeley, CA, USA (2003)

10. Fowler, M.: Inversion of Control – IoC (2004), http://martinfowler.com/articles/injection.html

11. G2CL: Generic Group Communication Layer (2009), http://g2cl.googlecode.com/

12. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Boston, MA, USA (1995)

13. Hedera: Hedera Group Communications Wrappers (2008), http://hederagc.sourceforge.net/

14. Jain, R.: The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley-Interscience, New York, NY, USA (1991)

15. JBoss: JBoss Application Server (2009), http://wwwJBoss.org/jbossas/

16. JGroups: JGroups – Building Blocks (2009), http://www.jgroups.org/blocks.html

17. JOnAS: JOnAS – Java Open Application Server (2009), http://jonas.ow2.org/

18. JXTA: JXTA Community Project (2008), https://jxta.dev.java.net/

19. Lodi, G., Panzieri, F., Rossi, D., Turrini, E.: SLA-Driven Clustering of QoS-Aware Application Servers. IEEE Transactions on Software Engineering 33(3), 186–197 (2007)

20. Miranda, H., Pinto, A., Rodrigues, L.: Appia – a Flexible Protocol Kernel Supporting Multiple Coordinated Channels. In: Proceedings of the 21st International Conference on Distributed Computing Systems (ICDCS’01). pp. 707–710. IEEE CS Press, Phoenix (Mesa), Arizona, USA (2001)

21. van Nieuwpoort, R.V., Kielmann, T., Bal, H.E.: User-Friendly and Reliable Grid Computing Based on Imperfect Middleware. In: Proceedings of the ACM/IEEE Conference on Supercomputing (SC’07). ACM Press, Reno, Nevada, USA (2007)

22. Pietzuch, P., Eyers, D., Kounev, S., Shand, B.: Towards a Common API for Publish/Subscribe. In: Proceedings of the 2007 Inaugural International Conference on Distributed Event-based Systems. pp. 152–157. ACM, Toronto, Ontario, Canada (2007)

23. PostgreSQL: PostgreSQL (2009), http://www.postgresql.org/

24. Sales, L., Teófilo, H., D’Orleans, J., Mendonça, N.C., Barbosa, R., Trinta, F.: Performance Impact Analysis of Two Generic Group Communication APIs. In: Proceedings of the 1st IEEE International Workshop on Middleware Engineering (ME’09). pp. 148–153. IEEE CS Press, Bellevue, WA, USA (2009)

25. Sales, L., Teófilo, H., Mendonça, N.C., D’Orleans, J., Barbosa, R., Trinta, F.: An Evaluation of the Performance Impact of Generic Group Communication APIs. Int. Journal of High Performance Systems Architecture 2(2), 90–98 (2009), DOI: http://dx.doi.org/10.1504/IJHPSA.2009.032026

26. Shoal: Shoal – A Dynamic Clustering Framework (2008), https://shoal.dev.java.net/

27. SUN: Java platform, enterprise edition (java ee) (2006), http://java.sun.com/javaee/

28. Triola, M.F.: Elementary Statistics. Addison-Wesley, Boston, MA, USA, 7th edn. (1997)