Package: oar-docker
Build your own OAR cluster with dockeroar-docker is a set of docker images especially configured for deploying
your own OAR cluster. The main idea is to have a mini development cluster with
a frontend, a server and some nodes that launch in just a few seconds on a
simple laptop.Why use oar-docker?Various case scenarios may affect you:Quickly test OAR on a clusterGain time: a ten-node cluster (or more) is launched in just a
few seconds and is cleaned in less than a second.Save resources: docker allows the user to pool the node between
various systems, resource utilization is thus considerably reduced.Synced volume : allowing you to continue working on your host machine, but
use the resources in the guest machine to compile or run your project.InstallationRequirements:python >= 2.7docker >= 1.3cgroup v1You can install, upgrade, uninstall oar-docker with these commands:$ pip install [--user] oar-docker
$ pip install [--user] --upgrade oar-docker
$ pip uninstall oar-dockerOr from git (last development version):$ pip install git+https://github.com/oar-team/oar-docker.gitOr if you already pulled the sources:$ pip install path/to/sourcesUsing a virtualenv may help overcome issues between python and your distribution.UsageUsage: oardocker [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

  Manage a small OAR developpement cluster with docker.

Options:
  --workdir DIRECTORY   Changes the folder to operate on.
  --docker-host TEXT    The docker socket [default:
                        unix://var/run/docker.sock].
  --cgroup-path TEXT    The cgroup file system path [default: /sys/fs/cgroup].
  --docker-binary TEXT  The docker client binary [default: docker].
  --verbose             Verbose mode.
  --debug               Enable debugging
  --version             Show the version and exit.
  -h, --help            Show this message and exit.

Commands:
  build    Build base images
  clean    Remove all stopped containers and untagged...
  connect  Connect to a node.
  destroy  Stop containers and remove all images
  exec     Run a command in an existing node.
  init     Initialize a new environment.
  install  Install and configure OAR from src
  logs     Fetch the logs of all nodes or only one.
  reset    Restart the containers
  start    Create and start the nodes
  status   Output status of the cluster
  stop     Stop and remove all nodesGetting startedTo get started with oar-docker, the first thing to do is to initialize a
project:$ oardocker initIf you already have OAR sources, the best is to initialize directly the
oardocker project in the OAR sources directory:$ cd path/to/oar/src
$ oardocker initYou have to do this only once. It allows you to import the Dockerfiles
and other configuration files.We then launch the base image build:$ oardocker buildNow, we have to install OAR. To do this, several options are available.If you already have the OAR sources:$ oardocker install /path/to/oar_srcOr if you want to install from tarball:$ oardocker install http://oar-ftp.imag.fr/oar/2.5/sources/testing/oar-2.5.4+rc4.tar.gzYou can also launch the installation from a git repository:$ oardocker install git+https://github.com/oar-team/oar.gitWe start a OAR cluster with 5 nodes:$ oardocker start -n 5It is possible to share directories between host machines and
all containers with the-voption:$ oardocker start -v $PWD:/oar_src -v /my/custom/lib:/usr/local/ma_libTo manage the cluster:$ oardocker connect frontend|server|nodeXX
$ oardocker logs [frontend|server|nodeXX]To clean:$ oardocker stop  # stops and removes all containers
$ oardocker clean  # removes all stopped containers (failed) and the untagged images <none:none>
$ oardocker destroy  # removes all images and containersWith oar-docker, it is possible to chain all commands to go faster:$ oardocker init -f build install oar-2.5.4+rc4.tar.gz start -n 4 connect -l root frontendFor instance, to develop on OAR, we often need to install OAR,
start the cluster and connect to it:$ oardocker install $PWD start -n 10 -v $PWD:/home/docker/oar_src connect frontendOne last thing to know. Thestopcommand is automatically launched before
everystart,installandbuild… If we launch multiple times the
last command, we will always obtain the same result. It can be useful to
experiment and develop (even) faster.Note for OAR < 2.5.9+g5k5oardocker ≥ 1.6.0 does not create the/dev/oar_cgroups_links/and/dev/cpusetinoardocker install, to let OAR take care of it so that the concerned part ofjob_resource_manager_cgroup.plis actually tested.This breaksjob_resource_manager_cgroup.plbefore OAR 2.5.9+g5k5. See.oardocker/init-scripts/prepare_oar_cgroup.shto revert to the old behaviour, by setting:CREATE_OAR_CGROUPS_LINKS=yesThe oardocker on Debian setupTo use oardocker on a Debian 11 (bullseye) system, one must activate Linux cgroup v1. This can be achieved by including in the linux kernel cmdline:systemd.legacy_systemd_cgroup_controller=true(add those options to your bootloader configuration).Using a python3 venvoardocker can be installed in a python3 venv:$ python3 -m venv oardocker
$ cd oardocker
$ . bin/activate
$ pip install oar-docker
$ oardocker init -e bullseye
$ oardocker build
$ oardocker install path/to/oar/git/repository
$ oardocker start
$ oardocker connect
$ ...Network servicesBy default, oardocker forwards the 80 and 6667 TCP ports to the frontend
container, for the OAR web services. To add other TCP ports forwarding to
the frontend, modify the.oardocker/manifest.jsonfile, adding extra lines
in thenet_servicesarray. For instance:"network_services": [
    ["Netcat", " (tcp test)", "5000", "TCP: "],
    ["Secure web service", "/secure", "443", "https://"],
    ["Python API", "/newoarapi", "6668"],
    ...
]Which translates to the following forwardings:******************** Network Services *********************

              Netcat: TCP: localhost:45000 (tcp test)
  Secure web service: https://localhost:40443/secure
          Python API: http://localhost:46668/newoarapi
                 ...
***********************************************************Only the numerical port numbers really matters, texts are only informative.Securityoar-docker is a development project and a testing one. It is in no way secure.
Besides, the private ssh key used is also insecured since it is public (you can
find it in the sources).oar-docker CHANGELOGVersion 1.6.4Released on April 17th 2023
- add bookworm support
- rework image templates
- switch to python3 venvVersion 1.6.3Released on January 06th 2022
- add bullseye supportVersion 1.6.2Released on October 21st 2021
- update docker image recipes for Debian buster: use buster instead of stable as the release name, add the man package
- add support for the oar-node and oar-server systemd native unit files
- fix python3 collections callableVersion 1.6.1Released on March 31st 2020
- Set the drawgantt label_cmp_regex configuration to sort nodes correctlyVersion 1.6.0Released on March 31st 2020Change theweb_servicestonet_servicesin themanifest.jsonfile (but keep the backward compatibility)Add an extra field in thenet_servicesfor port forwarding other than http (cosmetic)Add information in theREADME.rstfile about the TCP ports forwardingDo not create the/dev/oar_cgroups_links/and/dev/cpusetinoardocker install, but let OAR take care of itThis breaks with OARjob_resource_manager_cgroup.plbefore OAR 2.5.9+g5k5, seeREADME.rstVersion 1.5.0Released on March 06th 2020Version 1.4.0Released on December 06th 2018Improve systemd usage in containerUse of Docker API version 1.21EnhanceChange Debian8 base imageAdd better CiGri templateVersion 1.3.0Released on June 13th 2016Rework install_oar.sh scripts (apache2 configuration + chmod 0600 oar.conf)Add newapi apache2 config (for the Python OAR API)Fix apache2 startup by systemdRework port bindings, using the manifestAdd port bindings to 6668 for OAR APIChange default host port numbers for forwardingVersion 1.2.0Released on March 30th 2016Reverted the frontend http server from nginx to apacheFixed OAR Rest API unit testsConfigured COORM images to use the new oar3 python package and kamelot as default schedulerFixed “core” resources creationMinor python3 fixesVersion 1.1.0Released on February 10th 2016Updated base image version to 1.3.2Configured debian branches to pin some up-to-date packages from sid (nginx, systemd)Unmask systemd-tmpfiles-setup service (Fixed #45)Fixed /etc/hosts again mountpoint with the latest version of docker (>1.9)Added new coorm env based on jessie oneMade init-scripts executableThe install operation is not supported if no install_script is set to the manifest.jsonAdded a manifest.json file to describe environmentsTry to pull docker images if missingAdded--rebuildoption to rebuild images even if already donesend original oar-server log to journalctlAllocate tty by default inoardocker execcmdVersion 1.0.0Released on November 18th 2015Improved stability, performance and securityUsed systemd as default init for the containersImproved ressources usage with systemd activation socket.Used tmpfiles.d to create runtimes volatile files (pid,/var/run etc.)Passed environement variables to containers by using /etc/profileNo more insecure ssh keysFixed all web services (api, monika…) by replacing apache2 by nginxImproved logging by adding rsyslog node to centralize all logsAdded--no-tailand--lines/-noptions tooardocker logscommandCreated resources manually as it is faster than oar_resources_ini (no ssh connection)Removed unused scriptsOAR3 readyVersion 0.6.0Released on October 22nd 2015Removed wheezy environement (Fixed #39)Logged to stdout by defaultFixed oar.conf permissions to allow normal user to read itdisable_unicode_literals_warning in clickAdded oidentd start for the API to do authEnabled mod_cgi (for monika)Let the oar makefile configure the web toolsNew template for cigri baseFixed resources initializationFixed string formatingDo not allocate a pseudo-TTY by defaultFixed oar-node and oar-server init.d script for OAR 2.5.3 and olderAdapt new oar_resources_init optionsSetup cosystem and deploy jobs and install oar-node on the frontendVersion 0.5.10Released on July 03rd 2015Make /etc/oar/job_resource_manager_cgroup.pl a symlink to improve debugging (Fixed #34)Fixed API by reverted to oidentdVersion 0.5.9Released on July 01st 2015Set OAREXEC_DEBUG_MODE=1 to improve the dev environement (Fixed #34)Version 0.5.8Released on June 29th 2015Removed compiled python3 versionsBumped base images version to 1.2Version 0.5.7Released on June 25th 2015Bumped base images version to 1.1 (included apache2-suexec and pidentd)Fixed the stamp for setup_resources script (Fixed #33)Fixed oar-api apache configurationVersion 0.5.6Released on June 23rd 2015Used jessie as default envVersion 0.5.5Released on June 12th 2015Minor bug fix about persistent bash historyVersion 0.5.4Released on June 02nd 2015Bumped base image version to 1.0.4Added persistent .bash_history and .pyhistoryAdded :ro, :rw and :cow options to--volumeoptionFixed phppgadmin and oarapi 403 error in jessieFixed oarapi 403 error in jessieRemoved duplicated package installation from dockerfileFixed rest-client installation in debian jessieInstalled chandler in base imageConfigured postgresql just after OAR installationUpdate wait_pgsql script : used UNIX socket if no host providedVersion 0.5.3Released on May 22nd 2015Installed ruby-rspec librestclient-ruby for Rest API unittestsFixed oar resources initializationInitialized database during OAR installationInstalled chandlerSequential oar resources Initialization (Fixed #28)Run cleanup scripts and kill all processes in the container when receiving SIGINT (Fixed #27)Updated base images to version 1.0.3Improved oardocker cgroup cleanupCleanup oardocker nodes cgroup on oardocker stop (Fixed #27)Configured oarsh to get the current cpuset from the containers (Fixed #30)Added ugly patch to fix /etc/hosts mount with docker >=1.6.0Version 0.5.2Released on May 05th 2015Installed socat in the nodesWait ssh daemon on nodes before oar_resources_initImproved ssh connection on colmet nodesVersion 0.5.1Released on April 21st 2015Fixed compatibility with docker-py==1.1.0Version 0.5.0Released on Apr 2nd 2015Removed chandler and ruby from imagesInstalled libdatatime-perl on serverDropped python environmentAdded new environment for colmet based on jessie oneBased on oardocker/debian7 and oardocker/debian8 images built wit kameleonVersion 0.4.3Released on Feb 23rd 2015Added –debug optionSet default docker API to 1.15 (#25)Workaround phpphadmin apache installRemoved drawgantt-svg permissions errors (#23)Fixedoardocker initsubcommand (#22)Upload workdir to containers during the buildUpdated Dockerfiles to execute custom_setup.sh scriptVersion 0.4.2Released on Jan 28th 2015Cleaned up unversionned OAR files (git clean) from sources before installing OAR (Fixed #20)Version 0.4.0Released on Jan 24th 2015Python3 supportPrefixed all container outputs with the container hostname  (like oardocker logs subcommand)Added--force-rmand--pulloptions to oardocker build subcommandAllowed user to build custom images with custom_setup.sh script located in.oardocker/images/<image_name>/Added a proper way to shutdown containerUpdated /etc/hosts when reseting containersRemoved dockerpty package from dependenciesRemoved oardocker ssh/ssh-config subcommandAdded--verboseoptionFixed oardocker logs subcommandVersion 0.3.2Released on Dec 16th 2014Added--enable-x11option to allow containers to display x11 applicationsAuto-loaded OAR module on python startupAdded--envoption tooardocker startto set custom environment variablesAdded--workdiroption tooardocker execVersion 0.3.1Released on Nov 27th 2014Bug fixes:
- Fixed the Dockerfiles “FROM” statementImprovements:
- Removed implicit ‘default’ alias from available envVersion 0.3.0Released on Nov 27th 2014Features:Addedoardocker execcommandManage multiple environment variants withoardocker init: added wheezy|jessie|python bases imagesBug fixes:
- Revert default environment to Debian Wheezy due to breaking OAR API in Jessie
- Fixed locales issueImprovements:
- better synchronisation between oar-server and postgresql servicesVersion 0.2.0Released on Nov 5th 2014Features:Updated base images to debian jessieAddedoardocker connectto connect to the nodes without sshThe commandsoardocker sshandoardockerssh-configare deprecated from nowImprovements:Removed supervisor and make init process less complex by only using my_init.d statup scriptsCustomized help parameter to accept-hand--helpUsed docker client binary for some task instead of the APIBug fixes:Make sure that /etc/hosts file contain the localhost entryVersion 0.1.4Released on Oct 28th 2014Ignored my-init scripts if filename ends by “~”Added wait_pgsql script to wait postgresql to be availableFixed monika config (db server hostname is server)Removed old codeAdapt cgroup mount script to job_resource_manager_cgroup.pl and remove old cpuset workaroundFixed cpu/core/thread affinityVersion 0.1.3Released on Sep 10th 2014Addedoar resetcmd to restart containersAdded a better comments about oardocker images with git informationUsed default job_resource_manager script (from oar sources)Mount the host cgroup path in the containers (default path is /sys/fs/cgroup)Removed stopped containers from ssh_configRemove dnsmasq and mount a custom /etc/hosts for the nodes (need docker >= 1.2.0)Version 0.1.2Released on Sep 16th 2014Keep compatible with older versions of gitDon’t name the containersMounting OAR src as Copy-on-Write directory with unionfs-fuseStopped installation when container failed duringoardocker installAdded option to print versionAllow ssh connection with different userVersion 0.1.1Released on Sep 11th 2014Minor bug fixesVersion 0.1Released on Sep 11th 2014Initial public release of oar-docker