Package: os-faults
OpenStack fault-injection libraryThe library does destructive actions inside an OpenStack cloud. It provides
an abstraction layer over different types of cloud deployments. The actions
are implemented as drivers (e.g. DevStack driver, Fuel driver, Libvirt driver,
IPMI driver, Universal driver).Free software: Apache licenseDocumentation:https://os-faults.readthedocs.io/Source:https://opendev.org/performa/os-faults/Bugs:https://bugs.launchpad.net/os-faultsInstallationRequirementsAnsible is required and should be installed manually system-wide or in virtual
environment. Please refer to [https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html]
for installation instructions.Regular installation:pip install os-faultsThe library contains optional libvirt driver [https://pypi.org/project/libvirt-python/], if you plan to use it,
please use the following command to install os-faults with extra dependencies:pip install os-faults libvirt-pythonConfigurationThe cloud deployment configuration is specified in JSON/YAML format or Python dictionary.The library operates with 2 types of objects:service- is a software that runs in the cloud, e.g.nova-apicontainer- is a software that runs in the cloud, e.g.neutron_apinodes- nodes that host the cloud, e.g. a server with a hostnameExample 1. DevStackConnection to DevStack can be specified using the following YAML file:cloud_management:driver:devstackargs:address:devstack.localauth:username:stackprivate_key_file:cloud_keyiface:enp0s8OS-Faults library will connect to DevStack by addressdevstack.localwith userstackand SSH key located in filecloud_key. Default networking interface is specified with
parameteriface. Note that user should have sudo permissions (by default DevStack user has them).DevStack driver is responsible for service discovery. For more details please refer
to driver documentation:http://os-faults.readthedocs.io/en/latest/drivers.html#devstack-systemd-devstackmanagementExample 2. An OpenStack with services, containers and power managementAn arbitrary OpenStack can be handled too with help ofuniversaldriver.
In this example os-faults is used as Python library.cloud_config={'cloud_management':{'driver':'universal',},'node_discover':{'driver':'node_list','args':[{'ip':'192.168.5.127','auth':{'username':'root','private_key_file':'openstack_key',}},{'ip':'192.168.5.128','auth':{'username':'root','private_key_file':'openstack_key',}}]},'services':{'memcached':{'driver':'system_service','args':{'service_name':'memcached','grep':'memcached',}}},'containers':{'neutron_api':{'driver':'docker_container','args':{'container_name':'neutron_api',}}},'power_managements':[{'driver':'libvirt','args':{'connection_uri':'qemu+unix:///system',}},]}The config contains all OpenStack nodes with credentials and all
services/containers. OS-Faults will automatically figure out the mapping
between services/containers and nodes. Power management configuration is
flexible and supports mixed bare-metal / virtualized deployments.First let’s establish a connection to the cloud and verify it:cloud_management=os_faults.connect(cloud_config)cloud_management.verify()The library can also read configuration from a file in YAML or JSON format.
The configuration file can be specified in theOS_FAULTS_CONFIGenvironment
variable. By default the library searches for fileos-faults.{json,yaml,yml}in one of locations:current directory~/.config/os-faults/etc/openstackNow let’s make some destructive action:cloud_management.get_service(name='memcached').kill()cloud_management.get_container(name='neutron_api').restart()Human APIHuman API is simplified and self-descriptive. It includes multiple commands
that are written like normal English sentences.Service-orientedcommand performs specifiedactionagainstserviceon
all, on one random node or on the node specified by FQDN:<action> <service> service [on (random|one|single|<fqdn> node[s])]Examples:Restart Keystone service- restarts Keystone service on all nodes.kill nova-api service on one node- kills Nova API on one
randomly-picked node.Container-orientedcommand performs specifiedactionagainstcontaineron all, on one random node or on the node specified by FQDN:<action> <container> container [on (random|one|single|<fqdn> node[s])]Examples:Restart neutron_ovs_agent container- restarts neutron_ovs_agent
container on all nodes.Terminate neutron_api container on one node- stops Neutron API
container on one randomly-picked node.Node-orientedcommand performs specifiedactionon node specified by FQDN
or set of service’s nodes:<action> [random|one|single|<fqdn>] node[s] [with <service> service]Examples:Reboot one node with mysql- reboots one random node with MySQL.Reset node-2.domain.tld node- resets nodenode-2.domain.tld.Network-orientedcommand is a subset of node-oriented and performs network
management operation on selected nodes:<action> <network> network on [random|one|single|<fqdn>] node[s]
    [with <service> service]Examples:Disconnect management network on nodes with rabbitmq service- shuts
down management network interface on all nodes where rabbitmq runs.Connect storage network on node-1.domain.tld node- enables storage
network interface on node-1.domain.tld.Extended API1. Service actionsGet a service and restart it:cloud_management=os_faults.connect(cloud_config)service=cloud_management.get_service(name='glance-api')service.restart()Available actions:start- start Serviceterminate- terminate Service gracefullyrestart- restart Servicekill- terminate Service abruptlyunplug- unplug Service out of networkplug- plug Service into network2. Container actionsGet a container and restart it:cloud_management=os_faults.connect(cloud_config)container=cloud_management.get_container(name='neutron_api')container.restart()Available actions:start- start Containerterminate- terminate Container gracefullyrestart- restart Container3. Node actionsGet all nodes in the cloud and reboot them:nodes=cloud_management.get_nodes()nodes.reboot()Available actions:reboot- reboot all nodes gracefullypoweroff- power off all nodes abruptlyreset- reset (cold restart) all nodesdisconnect- disable network with the specified name on all nodesconnect- enable network with the specified name on all nodes4. Operate with nodesGet all nodes where a service runs, pick one of them and reset:nodes=service.get_nodes()one=nodes.pick()one.reset()Get nodes where l3-agent runs and disable the management network on them:fqdns=neutron.l3_agent_list_hosting_router(router_id)nodes=cloud_management.get_nodes(fqdns=fqdns)nodes.disconnect(network_name='management')5. Operate with servicesRestart a service on a single node:service=cloud_management.get_service(name='keystone')nodes=service.get_nodes().pick()service.restart(nodes)6. Operate with containersTerminate a container on a random node:container=cloud_management.get_container(name='neutron_ovs_agent')nodes=container.get_nodes().pick()container.restart(nodes)License notesAnsible is distributed under GPL-3.0 license and thus all programs
that link with its code are subject to GPL restrictions [1].
However these restrictions are not applied to os-faults library
since it invokes Ansible as process [2][3].Ansible modules are provided with Apache license (compatible to GPL) [4].
Those modules import part of Ansible runtime (modules API) and executed
on remote hosts. os-faults library does not import these module
neither static nor dynamic.[1]https://www.gnu.org/licenses/gpl-faq.html#GPLModuleLicense[2]https://www.gnu.org/licenses/gpl-faq.html#GPLPlugins[3]https://www.gnu.org/licenses/gpl-faq.html#MereAggregation[4]https://www.apache.org/licenses/GPL-compatibility.html