Package: ombt-orchestrator
Message bus evaluation frameworkContextThis is a framework to benchmark the communication middleware supported byoslo.messaging. It's primary goal is to address the evaluation ofhttps://docs.openstack.org/performance-docs/latest/test_plans/massively_distribute_rpc/plan.html.It is build on top of:EnOSlib. This library helps to describe the experimental workflow and enforce it : from the deployment to the performance metrics analysis.ombt. This will coordinate the benchmark once all the agents are up and running.From a high level point of view the framework is able to deploya communication bus (e.g RabbitMQ, qdr aka qpid-dispatch-router),a set of client/server that will communicatestart a benchmark while gathering metricsA typical test consists in the following components:Client 1---------+      +----------------------+     +-----> Server 1
                 |      |                      |     |
                 +----> |  Communication       | ----+-----> Server 2
Client 2--------------> |  Middleware          |     |
                 +----> |  (e.g qdr, rabbitms) |     |
...              |      |                      |     |
                 |      +----------------------+     +------> Server n
Client n---------+              |                             /
  \                                                         /
    \                           |                         / 
      \  --  --  --  --  -- Monitoring --  --  --  --  --InstallationClone the repository:git clone https://github.com/msimonin/ombt-orchestrator
cd ombt-orchestratorInstall the dependenciespip install -U pip
pip install -e .On Grid'5000 you can launch this command from any frontend.ConfigurationThe default configurations are currently defined in theconf.yamlfile.Command line interface> oo
Usage: oo [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  backup       Backup environment logs [after test_case_*].
  campaign     Perform a TEST according to the (swept)...
  deploy       Claim resources from a PROVIDER and configure...
  destroy      Destroy all the running dockers (keeping...
  g5k          Claim resources on Grid'5000 (frontend).
  inventory    Generate the Ansible inventory [after g5k,...
  prepare      Configure available resources [after g5k,...
  test_case_1  Run the test case 1: one single large...
  test_case_2  Run the test case 2: multiple distributed...
  test_case_3  Run the test case 3: one single large...
  test_case_4  Run the test case 4: multiple distributed...
  vagrant      Claim resources on vagrant (localhost).Workflow to run a test caseDeploying and launching the benchmark (default driverbrokeris defined in the configuration file)# default confs.yaml on $PWD will be read
> oo deploy --driver=broker vagrant

# Launch the one benchmark
> oo test_case_1 --nbr_clients 10 --nbr_servers 2Adapt to the relevant provider (e.gg5k)Real-time metrics visualisationGrafana is available on the port 3000 of the control node (check the inventory file).Backuping the environment> oo backupThe files retrieved by this action are located incurrent/backupdir by default.Some cleaning and preparation for the next run# Preparing the next run by cleaning the environment
> oo destroy
> oo deploy vagrant

# Next run
> oo test_case_1 --nbr_clients 20 --nbr_servers 2It's possible to force an experimentation dir with--env mydirNote also that scripting from python is also possible using the function defined intask.py## Workflow to run a campaignA campaign is a batch execution of several configurations for a given test case.
Deployment and execution of a benchmark is read from a configuration file. For example,
to run the first test case enabled on the framework run:>oocampaign--providerg5ktest_case_1Alternatively a campaign can be executed in aincrementalmode in which deployments are
performed only when a differentdriverorcall_typeis defined. Incremental campaigns
are executed with a different semantic on the parameters defined in the configuration.
With the incremental option the semantics is based on the combination of parameters by
means of a dot product between a set of them in the configuration file (i.e., azipoperation between the lists of parameters). These parameters are defined by test case
as follows:Test case 1:nbr_clients,nbr_serversandpauseTest case 2:nbr_topicsandpauseTest case 3:nbr_clients,nbr_serversandpause(onlyrpc-castcalls)Test case 4:nbr_topicsandpause(onlyrpc-castcalls)To execute an incremental campaign be sure to use the ombt versionmsimonin/ombt:singletoninstead of the default and execute:>oocampaign--incremental--providerg5ktest_case_1Misc.Bound clients or servers to specific bus agents:To bind ombt-clients to a specific bus instance you can declare the followingroles: [bus, bus-client].Following the same idea ombt-servers can be bound to a specific bus instance usingroles: [bus, bus-server]