GEMBus based Services Composition
Platform for Cloud PaaS
Yuri Demchenko, University of Amsterdam
on behalf ofYuri Demchenko, Canh Ngo, Cees de Laat
Pedro Martínez-Julia, Elena Torroglosa, Antonio D. Perez-Morales
Mary Grammatikou, Jordi Jofre, Steluta Gheorghiu, Joan A. Garcia-Espin

ESOCC2012, 19-21 September 2012
Bertinoro, Italy
connect • communicate • collaborate

Outline
GEMBus – GEANT Multi-domain service Bus
Composable Services Architecture (CSA) as a GEMBus architecture basis
GEMBus as a Platform for Cloud based service provisioning
GEMBus components and core services
Composition Service, Secure Token Services (STS), Accounting,Repository, Registry
GEMBus testbed and demonstrators
Collaborative development platform for GEANT/NREN community
Future developments
Contribution to OGF ISOD-RG activity, other community activities

connect • communicate • collaborate

GEMBus (GEANT Multi-domain Bus)GEMBus (GEANT Multi-domain Bus) is a service-oriented middleware
platform that allows flexible services composition and their on-demand
provisioning to create new specialized task-oriented services andapplications
Part of EU Project GN3 JRA3 Task 3 “Composable Services”
GEMBus is built upon state-of-the-art Enterprise Service Bus (ESB)
technology and extends it with new functionalities that allow dynamic
component services composition, deployment, and management
Implements the Composable Services Architecture (CSA)
Evolves as a cloud Platform as a Services (PaaS)
Cloud based services virtualisation
Community oriented
Provides core GEMBus/PaaS services to support multi-domain services
and users federation, including network connectivity
eduGAIN, Identity federation, federated network access(GEANT based)
connect • communicate • collaborate

Inter-domain Services
Integration (Federation)connect • communicate • collaborateComposable Service Architecture(CSA)
Composable Services Architecture (CSA) provides a basis for flexible
services composition, integration and operation of existing component
services
Defines functionalities related to Control and Management planes,
allowing the integration of existing distributed applications andprovisioning systems
Based on the virtualisation of component services
Supports (composable) Services Lifecycle Management (SLM)(1) Service Request, (2) Reservation, (3) Deployment, (4)Operation, (5) Decommissioning
Provides a basis for defining GEMBus middleware
Leverages the SOA ESB architecture and Web Services platformconnect • communicate • collaborate

Composable Services Architecture

Applications and User Terminals

User
ClientAdaptation Layer – Composite/Virtualised Services and Resources

Control &Management
Plane(Operation,Orchestration)

Composition
Layer(Reservation
SLA
Negotiation)

Composable Services Middleware(GEMBus)MD SLC

Registry

Logging

Security

Logical Abstraction Layer for Component
Services and Resources

Adaptation Layer - Component Services and Resources

StorageResources

ComputeResources

Network Infrastructure

Component Services & Resources
Control links

Data links/flows
connect • communicate • collaborate

Composable Services
Lifecycle/Provisioning Workflow
Main stages/phases
Service Request/
SLA Negotiation

Composition/Reservation
Re-Composition

ServiceLifecycle
MetadataService(SL MD)

Deployment

Additional stages

Registr&Synchro
Recovery/
Migration

Operation(Monitoring)

Decommissioning

Service Request (including SLAnegotiation)
Composition/Reservation (akadesign)
Deployment, including
Reqistration/Synchronisation
Operation (including Monitoring)
Decommissioning

Provisiong
SessionManagnt

Re-Composition should addressincremental infrastructurechanges
Recovery/Migration can use SLMD to initiate resources resynchronisation but may require
re-composition

The whole workflow is supportedby the Service Lifecycle Metadata
Service (SL MD)
connect • communicate • collaborate

Slide_7

Composable Services Architecture
Lifecycle stages workflow
4ServUserClient4 34Serv

Applications and User Terminals

Serv(1) Request(2) Composition/Reservation(3) Deployment(4) Operation(5) Decommissioning

Proxy (adaptors/containers) – Composed/Virtualised Services and Resources

132

Control &Management
Plane

1

(Operation,Orchestration)

42
3Composition
Layer /Serv

Composable Services Middleware(GEMBus/GESB)(ReservationSLA
Negotiatn)

2
MD SLC

Registry

Logging

Composable Services
lifecycle/provisioning
stagesMD SLC – Service
Lifecycle Metadata

Security

GEMBus – GEANT
Multidomain Bus

3 2

Logical Abstraction Layer for Component
Services and Resources

GESB – Geysers ESB

3 2443 2

Proxy (adaptors/containers) - Component Services and Resources

StorageResources

ComputeResources

Network Infrastructure
Component Services & Resources

Control/Mngnt Links
connect • communicate • collaborate
Data Links

Slide_8

Example Service Composition –Service NX
Service 4(CSrvID,SesID)

UserClient

Composed Service NX(CSrvID,SesID)

Service 3

Service 1

Service 2(CSrvID,SesID)(CSrvID,SesID)(CSrvID,SesID)

Role and place for
Composition andOrchestration* Composable
services or GEMBus
infrastructure service(CSrvID,SesID)

Service 4

Composed Service NX(CSrvID,SesID)(CSrvID,SesID)

User Interface

UserClient

Frontend
CompSrvNX(CSrvID,SesID)(CSrvID,SesID)

GN3-JRA3-T3
Discussion

Service 3

Service 1

Service 2(CSrvID,SesID)(CSrvID,SesID)(CSrvID,SesID)

Composable Services Architecture

CSrvID, SesID –bind component
services into the
on-demand
provisioned
Composedservice NX

connect • communicate • collaborate

Slide_9

GEMBus as a Service Bus forheterogeneous services Integration

connect • communicate • collaborateGEMBus Reference Container
Reference service template (orReference Container) is defined byan API that needs to be followed inorder to connect to GEMBusplatform.
Provides set of configurable
interceptors to GEMBus coreservices
Provided as a standard piece ofcode for developing new andadapting any external APIs toGEMBus
Provided as part of preconfigured
GEMBus VMs for cloud PaaS
services integration

connect • communicate • collaborate

GEMBus Core Services

Federated Service Registry: stores and provides information about
GEMBus services.
Service Repository: stores service bundles, thus allowing theirdeployment via GEMBus.
Composition Service: enables services composition and supported by
the orchestration engine.
Security Token Service (STS): issues, verifies and translates security
tokens to allow the authentication of requesters in a federated, multidomain environment.
Accounting Service: provides configurable and aggregated access tothe GEMBus login service to support monitoring, auditing, diagnostics,and troubleshooting.
Messaging service that provides underlying infrastructure forcomposable services interaction, integration and QoS management
GEMBUs Messaging Infrastructure (GMI)
connect • communicate • collaborate

Composition Service

Uses Business Process Execution Language (BPEL) and available
ESB-based workflow execution engines to enable services orchestration
Connected with a specific description and control tools based on
Business Process Modeling Notation (BPMN)– From the analysis of a business processes to implementationand management
First prototype implemented as an Eclipse plug-in and provides
mapping to the underlying BPEL execution language
Allows integration of other workflow engine and support otherworkflow execution languages
Orchestration service based on Taverna workflow management system
Cross-platform set of tools enabling users to compose web services
supporting also local Java services (Beanshell scripts), local Java
APIs, R scripts and data import from Excel or in CVS format
Moving to Intalio|BMPS platform – full compatibility
connect • communicate • collaborate

Example services composition withIntalio|BMPS designer

Intalio|BPMS Business Process Management System(BPMS)
Based on the open source Eclipse BPMN Modeler, Apache
ODE BPEL engine, and Intalio Tempo WS-Human Task
serviceconnect • communicate • collaborate

Security Token Service (STS)GEMBus Security Token Services (STS) implements the generic STS
as defined in WS-Security and WS-Trust as a security mechanism toconvey security information between services and supporting federated
services infrastructure
STS supports issue and validation of the basic security tokens such as
SAML, JWT, and X.509 certificatesSTS supports services (identity) federation and federated identity
delegation

GEMBus STS support token issues and validation
Ticket Translation Service (TTS) responsible for generating valid tokens inthe system according to the received credentials, renewing and converting
security tokens
Authorisation Service (AS) supporting token validation (on request from aservice) and can also retrieve additional attributes or policy rules from othersources to perform the validation

connect • communicate • collaborate

Security Token Service (STS)
operation in AuthN and AuthZ

connect • communicate • collaborate

GEMBus Accounting Service (1)Consists of a Common Accounting Repository and an Accounting Module (AM) deployed at every
GEMBus instance. The Accounting Module includes the following main building blocks:
Data Collection: raw log, event, state data collection from services at ESB level
Works asynchronously and triggered each time a service is being called
Data Storage: stores raw data provided by the Data collection module
Data Processing: filter/select data for computing specific metrics of interest related to eachservice
Can work both continuously and on request from other services
Performs aggregation of accounting data from all available domains
Information Reporting: produces reports
in a human-readable format
Can aggregate data from different
domains over the monitored
GEMBus infrastructure

connect • communicate • collaborate

GEMBus Accounting Service (2)connect • communicate • collaborate

GEMBus/ESB testbed

Fuse ESB (Servicemix and Fuse Fabric as adopted for clouds): service
deployment environment
Fuse Message Broker: connect message brokers in intra-domains andinter-domains
Fuse Mediation Router: configurable message routes in in composableservices
Java OSGi Bundles for service deployment
VM management platform
OpenNebula and VirtualBox

connect • communicate • collaborate

Testbed Architecture for GEMBus
based Cloud PaaS
Network Infrastructure (aka NaaS)

Service-1

Service-2
PaaS
Inter-ESB/
Inter-domain
messagingService-3

Service-4

Message
Router (Camel)

Message
Router (Camel)

Message
Router (Camel)

Message
Router (Camel)

Message
Broker(ActiveMQ)

MessageBroker(ActiveMQ)

MessageBroker(ActiveMQ)

MessageBroker(ActiveMQ)

GEMBus

GEMBus

GEMBus

GEMBus

VM

VM

VM

VM

connect • communicate • collaborate

Example Service composition
workflow(A v B v (A+B)) + C => Average service => Visualisation
Presented at SuperComputing 2011

connect • communicate • collaborate

Controller and Network of Brokers

httpd,maven-repo

VMController
GEMBusVM

GEMBus
GEMBusVM

VM
GEMBus

VM
Inter-GEMBus
Signal Link

GEMBus/VM
Control Link

Beans.xml
uri="jms:S1_Out"/>
to uri="jms:S3_Out"/>
uri="jms:S2_Out"/>
to uri="jms:S3_Out"/>
uri="jms:queue:S3_Out"/>
uri="bean:logger?method=log”/
Java DSL
from(“jms:S1_Out”).to(“jms:S3_In”);
from(“jms:S2_Out”).to(“jms:S3_In”);
from(“jms:S3_Out”).
bean(LogSignal.class,“log(${body})”);

connect • communicate • collaborate

Inter/multi-domain message
routing/brokering in GEMBus
B

B
B

B
BM

M

Master broker node toconnect administrative
domains

B

ESB broker instance tomediate local services

Domain 1Domain 3BM

BB

BM

Domain 2BB

B

B
B

B

connect • communicate • collaborate

GEMBus in a multi-layer
infrastructure services provisioning
Network topology/QoS aware
infrastructure services
provisioning
GEYSERS Projecttechnology
Services composition platform
for community oriented services
and/or applications
GEMBus@GN3 Project

connect • communicate • collaborate

General infrastructure provisioning:
Workflow => Logical (Cloud) Infrastructure (1)Enterprise/Scientific workflow
Storage
Data

SpecialProc 1
Data
Filtering

Input
Data

VisualPresentSpecial
Proc 2

Instrum.
Data

Campus A

Data
Archive

Visualisation

Visualisation

CE

User
Group A

Campus B

CE

User
User
User

UserUser
User
VR6VR2Cloud 2 PaaS

UserGroup B

VR7
VR4

VR1

VR5

Resource/
ServiceProvider

VR3

Enterprise/Project basedIntercloud Infrastructure

Cloud 1 IaaS

Resource/ServiceProvider
CNCN
CN

CN
CN

CN

Cloud PaaS Provider

CN

CN
CN

InterCloud Architecture
2012

CN

Cloud IaaS Provider
InterCloud Architecture
FrameworkCN

connect • communicate • collaborate

25

General infrastructure provisioning:
Workflow => Logical (Cloud) Infrastructure (2)Enterprise/Scientific workflow
Storage
Data

SpecialProc 1
Data
Filtering

Input
Data

VisualPresentSpecial
Proc 2

Instrum.
Data

Campus A

Data
Archive

Visualisation

Visualisation

CE

User
Group ACampus BCE

User
User
User

User
User
User
VR6
VR2Cloud 2 PaaS

UserGroup B

VR7
VR4

VR1VR5

Resource/ServiceProvider

VR3Enterprise/Project basedIntercloud Infrastructure

Cloud 1 IaaS

Resource/ServiceProvider
CN

CN
CNCN
CN

CN

Cloud PaaS Provider

CN

CN
CN

InterCloud Architecture
2012

CN

Cloud IaaS Provider
InterCloud Architecture
FrameworkCN

connect • communicate • collaborate

26

General infrastructure provisioning:
Workflow => Network Infrastructure
Resource and Cloud Provider Domains
Cloud 1 IaaS
VR3

VR1

Cloud 2 PaaS

VR5

VR7Campus A
Infrastructure

Campus B
Infrastructure

VR4VR2

Campus A

VR6

Cloud Carrier Network Infrastructure

Visualisation

Visualisation

CEUserGroup A

Campus B

CEUser
User
User

User
User
User
VR6
VR2

Cloud 2 PaaS

UserGroup B

VR7
VR4

VR1

VR5

Resource/
ServiceProvider

VR3Enterprise/Project basedIntercloud Infrastructure

Cloud 1 IaaS

Resource/
ServiceProvider
CN

CN
CN

CN
CN

CN

Cloud PaaS Provider

CN

CN
CN

InterCloud Architecture
2012

CN

Cloud IaaS Provider
InterCloud Architecture
FrameworkCN

connect • communicate • collaborate

27

Future work and Discussion

Promoting GEMBus as a platform Cloud PaaS to wider community
Research on Semantic services composition based on Semantic
Services description and semantic enabled Service Registry
Support REST/SOAP integration with OAuth2lib and GemSTS andmutual delegation
JSON (JavaScript Object Notation) and SOAP token mapping
Integration of the core GEANT network services
Authobahn – On-demand network connectivity provisioning
PerfSONAR – Network monitoring system (open source)
eduGAIN – Inter-federation Identity management and Access
control system (GEANT/NREN oriented)
eduPKI – Personal certificates management system forGEANT/NREN

connect • communicate • collaborate

Questions and Discussion

connect • communicate • collaborateSemantic Services composition

Services can be semantically described and populated via GEMBus Registrythat uses RDF and ontology to store service descriptions
This allows semantic searching with complex queries based on non-trivial
attributes and service relations
External ontologies can be used via Registry infrastructure federation andmapping
A composed service may be semantically described in terms of standard
ontology. The composition engine will get the semantic description and find thenecessary services in the GEMBus Registry by performing the semantic
searches.
Example: We have three services: A service (S1) that produces messages oftype A which may be sent as a flow. A transformation service that receives
messages of type A, transforms them to type B, and sends them out. And aservice (S2) that consumes messages of type B.
The composed service may be defined by using a simple description:
Connect S1 to S2
The composition engine will know that S1 and S2 are incompatible and willautomatically request a transformation service
connect • communicate • collaborate
from type A to type B.

Additional information

GEMBus namespace and services naming
First GEMBus/ESB demo (SC2011) details

connect • communicate • collaborate

GEMBus Messaging Infrastructure(GMI) Functionality
Provides configurable message handling and routing functionality forcomposed services
Supports GEMBus namespace and EPR/URI profiles
Service and security context handling, including services lifecycle
management– Context dependent rules to be definedPrimarily uses SOAP with future extension to REST
Currently REST doesn’t have well defined architecture model
Built on the top of the standard Apache/Fuse messaging infrastructure
Fuse Message Broker (Apache ActiveMQ) messaging processor
and cross-domain integration
Fuse Mediation Router (Apache Camel) normalised message router– Rule based message routing
Message level security and transport level security can be used as partof Fuse/Apache platform
connect • communicate • collaborate

GEMBus Namespace and naming
GEMBus uses designated namespace “urn:geant:gembus” and defines thefollowing initial namespace branches
urn:geant:gembus:protocol
urn:geant:gembus:service
urn:geant:gembus:security
All GEMBus defined names and attributes should have prefix “gembus”
Branches may use specific prefix in a form “gembus-{branch}”
All GEMBus identifiers should use FQN (Fully Qualified Name) format
urn:geant:gembus:service:{service_type}:{attribute_name}
GEMBus should allow export of external namespaces in a standard way
GEMBus namespace should be supported by (centralised) GEANT
namespace registryconnect • communicate • collaborate

UvA Testbed – Cloud IaaS and
GEMBus/CSA

Services

ServiceService

Network connectivity for services

Inter-ESB/
Interdomain
Messaging

MQ

MQ

NMR

NMR

ESB

VM

VMS OpenNebula

Planned for SC12

Service middleware
GEMBus/ESB

MQ
NMR
ESB

ESB

VM
VM

VM
VM

VMS ONeb

VMS ONeb

Hardware (multi-core) platform
connect • communicate • collaborate

UvA Testbed – phase 1.1
Cloud/Network Infrastructure Layer

VM

VMS OpenNebulaServiceServices

Service

Network Operating System (NOS) - Network connectivity for services

VM
VM

VM
VM

VMS ONeb

VMS ONeb

Hardware (multi-core) platform

Phase 1.1 to create and test network infrastructurewith dummy services

connect • communicate • collaborate

UvA Testbed – phase 1.2
GEMBus/ESB/CSA Service Layer

Services

ESB(Intra)

MQ
BrokerNMR

VM/PCInter-ESB
Messaging

MQ
Brkr
NMR

VM/PC
VM

Phase 1.2 to create and test inter-ESB
messagingServiceServiceNetwork connectivity for services

Service middleware
GEMBus/ESB

MQ
BrkrNMRVM/PCVM

connect • communicate • collaborateCurrent Demo setup - Topology

Signal_source_s1 (message producer) -> VM1-Broker (Jupiter) <---> VM2Broker (Neptune) -> signal_sink_s3 (message consumer)
S1 = a + bt, where t – time with step = 0.1
S2 = sin (t)S3 = S1 + S2
VM1 Jupiter - Signal sources S1 and S2Signal_source_s1 sends messages to message queue “jms:SignalS1”
Signal_source_s2 sends messages to message queue “jms:SignalS2”
Jupiter Message Broker registers with Neptune Message Broker
VM2 Neptune – Signal sink and Camel router
Camel Router – Receives messages from “jms:SignalS1_Out” and“jms:SignalS1_Out” and sends to “jms:SignalS3_In”
Signal_sink_s3 subscribes messages from a message queue“jms:SignalS3_In”
connect • communicate • collaborate

ActiveMQ Message Broker
Configuration<blueprint ...>
<broker ...>
<networkConnectors><networkConnector name="ncNeptue“
uri="static:(tcp://192.168.56.102:62001)“ duplex="true"></networkConnector></networkConnectors><transportConnectors><transportConnector name="openwire" uri="tcp://localhost:61616"/><transportConnector name="stomp" uri="stomp://localhost:61613"/></transportConnectors></broker><bean id="activemqConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory"><property name="brokerURL" value="tcp://localhost:61616" /></bean><service ref="pooledConnectionFactory"
interface="javax.jms.ConnectionFactory"><service-properties><entry key="name" value="localhost"/></service-properties></service></blueprint>
connect • communicate • collaborate

Camel NMR Router Configuration<beans ….><bean id="jms" class="org.apache.camel.component.jms.JmsComponent"><property name="connectionFactory"><bean class="org.apache.activemq.ActiveMQConnectionFactory"><property name="brokerURL" value="tcp://localhost:61616"/></bean></property></bean><camelContext xmlns="http://camel.apache.org/schema/spring"><!– Route for S1 messages --><route><from uri="jms:S1_Out"/><to uri="jms:incomingQueueS3"/></route><!– Route for S2 messages --><route><from uri="jms:S2_Out"/><to uri="jms:incomingQueueS3"/></route></camelContext></beans>
connect • communicate • collaborate

GEMBus Messaging Infrastructure(GMI) in overall GEMBus architecture
GEMBus Component Services
Service 1

Service 2

Service 3

Service 4(CSrvID,SesID)(CSrvID,SesID)(CSrvID,SesID)(CSrvID,SesID)

Message Handling

Routing

Configuration

Interceptors(AspOrient)ESB

GEMBus Messaging
Infrastructure (GMI)GEMBusRegistry

Composition &Orchestration

SecurityService

GEMBus Infrastructure Services

Logging andAccounting
Service

GEMBus provides common (dynamically)
configurable messaging infrastructure for
Composable services communication

connect • communicate • collaborate