Package: janitor-osp
JanitorJanitor is a Linux component to perform quota management tasks to anOpenstacktenant.Itis notan official Openstack's tool, and Janitor's code is wrapping Openstack python client libraries:python-openstackclientto manage authentication and as general APIpython-neutronclientto manage networkpython-novaclientto manage compute nodes and vmspython-glanceclientto manage cloud imagespython-cinderclientto manage volumes and block storageTable of contentMotivationQuick linksRunning in dockerInstall and usageSetup your dev environmentRunning janitor functional testsContributingrelease pypi packageBuild RPM and releaseDEMOQuick linksrepodescriptionGitsource code repositoryCoprRPM buildQuay.iocontainer image buildtest.pypipypi repo for testing python package releasepypipypi repo for prod python package releaseMotivationI have been using OpenStack continuously since 2014 as a resource to run workloads with the main purpose of
testing software. Manage a healthy Openstack tenant quota is vital when working with CI/CD and a large number of
automated pipelines which run 24/7. A long time ago I and my team were having a lot of headaches with one of internal
Openstack the cluster which at that time had a very limited quota and workload capacity but was important for the
test workflow. With the constant false-positive for test failing due to lack of public/floating IP, storage/volume,
and even virtual CPU when provisioning new VMs, we got the idea of implementing Janitor, a super simple script/codes
toautomate the clean-up task of deleting left-over virtual machines as well network and volume from an Openstack  tenant, all managed by API and remote calls.Install and Usagefrom container:docker pull quay.io/ecerquei/janitor
docker run --rm -it --network host --name janitor ecerquei/janitor /bin/bash

# copy your files
docker cp openrc.sh janitor:/home/janitor
docker cp whitelist.txt janitor:/home/janitor

bash-4.4$ janitor --help
bash-4.4$ janitor openstack --openrc openrc.sh --whitelist whitelist.txt --keystone v3seecontainerfrom pip:https://test.pypi.org/project/janitor-osp/https://pypi.org/project/janitor-osp/python3 -m venv venv
source venv/bin/activate
pip install pip --upgrade
pip install -r https://raw.githubusercontent.com/eduardocerqueira/janitor/master/requirements/production.txt
python3 -m pip install janitor --upgradeForFedora:fromCoprdnf install dnf-plugin-copr
dnf copr enable eduardocerqueira/janitor
dnf install janitoror if you prefer you can install the repo fromhttps://copr.fedorainfracloud.org/coprs/eduardocerqueira/janitor/and maybe is needed to disablegpgcheck=0[eduardocerqueira-janitor]
name=Copr repo for janitor owned by eduardocerqueira
baseurl=https://copr-be.cloud.fedoraproject.org/results/eduardocerqueira/janitor/epel-7-$basearch/
type=rpm-md
skip_if_unavailable=True
gpgcheck=0
gpgkey=https://copr-be.cloud.fedoraproject.org/results/eduardocerqueira/janitor/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1from local build::make rpm
dnf install rpmbuild/RPMS/x86_64/janitor-0.2-0.x86_64.rpmfor CentOS:requires openstack repo. check the latest repohttps://wiki.openstack.org/wiki/Release_Namingsudo yum install centos-release-openstack-newton.noarch
sudo yum install install rpmbuild/RPMS/x86_64/janitor-0.2-0.x86_64.rpmusing Janitor to clean up left-over virtual machines and release floating ips for Openstack tenant:source ~/mytenant-openrc.sh
janitor openstack --whitelist /tmp/whitelist.txt --keystone v3

# or passing the path for your openrc file
janitor openstack --openrc /tmp/mytenant-openrc.sh --whitelist /tmp/whitelist.txt --keystone v3listing history for your janitor:janitor historylocal dev environmentRequirements:Linux OStested on Fedora and CentOSPython3packages:# install packages needed to interact with make, build RPM, make release to Copr, and generate doc
sudo dnf install redhat-rpm-config rpm-build python3-devel gcc python3-devel python3-pip python3-wheel python3-setuptools, python3-sphinx copr-cli

# prep your python env
git clone git@github.com:eduardocerqueira/janitor.git
cd janitor
python3 -m venv venv
source venv/bin/activate
pip install -r requirements/devel.txt

# check janitor installation
python janitor/cli.py --help
openstack --openrc /home/ecerquei/osp/janitor-openrc.sh --whitelist /home/ecerquei/osp/janitor-whitelist.txt --keystone v3seerunning janitor functional testsalso you can explore themake tasksrunningmake.when runningmake docthe generated doc can be access at : file:///home/user/git/janitor/docs/build/html/index.htmlContributingAny idea, suggestions and pacthes are welcome to this project! Fork the project, make your code change, run the test locally
to ensure your changes are not breaking any functionality, remember to run the code static analysis before submitting your
PR and if needed openissues or discussionon this project.# make flake8running in dockerimage is available inquay janitor repositorydocker pull quay.io/ecerquei/janitor
docker run --rm -it --network host --name janitor ecerquei/janitor /bin/bash

# running janitor inside the container
bash-4.4$ janitor --helpto build a new docker image:docker build -f Dockerfile . -t ecerquei/janitorpushing new container to quay# terminal 1
docker run --rm -it --network host --name janitor ecerquei/janitor /bin/bash

# terminal 2
docker ps -l # and copy the container id 
docker commit aa20639aa618 quay.io/ecerquei/janitor
docker push quay.io/ecerquei/janitorrelease pypi packageensure you have permission, and the API token saved at~/.pypircto publish new release to pypi.increment the build updating the varVERSION=0.3fromMakefilemake rpm ( needed to generate the new version file )run commands below:source venv/bin/activate
python3 -m build

# for testpypi
python3 -m twine upload --repository testpypi dist/*

# for pypi
python3 -m twine upload dist/*package will be available at:https://test.pypi.org/project/janitor-osp/#historyhttps://pypi.org/project/janitor-osp/#historyBuild RPM and release$ make

Usage: make <target> where <target> is one of

clean     clean temp files from local workspace
doc       generate sphinx documentation and man pages
test      run functional/unit tests locally
tarball   generate tarball of project
rpm       build source codes and generate rpm file
srpm      generate SRPM file
build     generate srpm and send to build in copr
all       clean test doc rpm
flake8    check Python style based on flake8Running from your local machine, you can generate your own RPM running:$ make rpmif your environment is properly setup you should have your RPM at: /home/user/git/janitor/rpmbuild/RPMS/x86_64/janitor-0.0.2-0.x86_64.rpmjanitor is being built onFedora Copr https://copr.fedorainfracloud.org/coprs/eduardocerqueira/janitor/builds/running a new build you need to check your ~/.config/copr-fedora file and run:make buildBefore starting the release process, check your account permissions in Copr.$ make srpmcopy rpmbuild/SRPMS/janitor-0.0.2-0.src.rpm to janitor/coprpush janitor/copr to githubcopr-cliwill be used, installed bysudo yum/dnf install copr-cliand configure it.Request asBuilderfor projectsjanitor, wait until admin approves.$ copr-cli build janitorhttps://github.com/eduardocerqueira/janitor/raw/master/copr/janitor-0.0.2-0.src.rpmGo and grab a cup of tea or coffee, the release build will be come out soon ::# tag based builds: `https://copr.fedorainfracloud.org/coprs/eduardocerqueira/janitor/builds/`Copr and RPM release helpful Linkshttps://fedorahosted.org/copr/wiki/HowToEnableRepohttp://fedoraproject.org/wiki/Infrastructure/fedorapeople.org#Accessing_Your_fedorapeople.org_Spacehttps://fedorahosted.org/copr/wiki/UserDocs#CanIgiveaccesstomyrepotomyteammatehttps://copr.fedoraproject.org/api/DEMORunning the program without parameters:[ecerquei@dev ~]$ janitor
Usage: janitor [OPTIONS] COMMAND [ARGS]...

  Janitor clean-up the left-over in your Openstack tenant to know more check
  documentation running: man janitor

Options:
  --help  Show this message and exit.

Commands:
  history    show janitor history
  openstack  clean-up VM and public IP from Openstack tenantRunning the program with with parameters to make clean-up:[ecerquei@dev ~]$ janitor openstack --openrc /home/ecerquei/git/janitor/tests/test-openrc.sh --whitelist /home/ecerquei/git/janitor/tests/whitelist.txt --keystone v3
+---------------------+--------------+------------------+-----------------------------+--------------------------------------------+--------+----------------------+
|      TIMESTAMP      |    ACTION    |       NAME       |             IPs             |                   IMAGE                    | FLAVOR |   CREATED AT (UTC)   |
+---------------------+--------------+------------------+-----------------------------+--------------------------------------------+--------+----------------------+
| 2017-07-13 15:00:57 |   deleted    |    QA-DB-100     |        172.16.91.195        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:56:38Z |
| 2017-07-13 15:00:57 |   deleted    |    DEV-WEB-10    |        172.16.91.194        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:56:24Z |
| 2017-07-13 15:00:57 |   deleted    |    PROD-APP1     |        172.16.91.193        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:56:10Z |
| 2017-07-13 15:00:57 |   deleted    | test_123_456_789 |        172.16.91.192        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:55:58Z |
| 2017-07-13 15:00:57 |   deleted    |    test123456    |        172.16.91.191        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:55:47Z |
| 2017-07-13 15:00:57 | in whitelist |  jslave_centos   | 172.16.91.173, 10.8.187.160 |            paws-jslave-centos7             |   3    | 2017-07-13T01:02:58Z |
| 2017-07-13 15:00:57 | in whitelist | jslave_fedora25  | 172.16.91.172, 10.8.187.207 |            paws-jslave-fedora25            |   3    | 2017-07-13T01:02:08Z |
+---------------------+--------------+------------------+-----------------------------+--------------------------------------------+--------+----------------------+
+---------------------+---------+--------------+
|      TIMESTAMP      |  ACTION | FLOATING IP  |
+---------------------+---------+--------------+
| 2017-07-13 15:00:57 | deleted | 10.8.180.199 |
+---------------------+---------+--------------+

+---------------------+---------+--------------------------------------+
|      TIMESTAMP      |  ACTION |              VOLUME ID               |
+---------------------+---------+--------------------------------------+
| 2017-07-13 15:00:57 | deleted | 358be8d1-6d4a-4db7-973f-8369d4ff86f7 |
+---------------------+---------+--------------------------------------+Running the program with parameter to print history file:[ecerquei@dev ~]$ janitor history
+---------------------+--------------+------------------+-----------------------------+--------------------------------------------+--------+----------------------+
|      TIMESTAMP      |    ACTION    |       NAME       |             IPs             |                   IMAGE                    | FLAVOR |   CREATED AT (UTC)   |
+---------------------+--------------+------------------+-----------------------------+--------------------------------------------+--------+----------------------+
| 2017-07-13 15:00:57 |   deleted    |    QA-DB-100     |        172.16.91.195        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:56:38Z |
| 2017-07-13 15:00:57 |   deleted    |    DEV-WEB-10    |        172.16.91.194        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:56:24Z |
| 2017-07-13 15:00:57 |   deleted    |    PROD-APP1     |        172.16.91.193        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:56:10Z |
| 2017-07-13 15:00:57 |   deleted    | test_123_456_789 |        172.16.91.192        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:55:58Z |
| 2017-07-13 15:00:57 |   deleted    |    test123456    |        172.16.91.191        | CentOS-Atomic-Host-7.20160130-GenericCloud |   2    | 2017-07-13T18:55:47Z |
| 2017-07-13 15:00:57 | in whitelist |  jslave_centos   | 172.16.91.173, 10.8.187.160 |            paws-jslave-centos7             |   3    | 2017-07-13T01:02:58Z |
| 2017-07-13 15:00:57 | in whitelist | jslave_fedora25  | 172.16.91.172, 10.8.187.207 |            paws-jslave-fedora25            |   3    | 2017-07-13T01:02:08Z |
+---------------------+--------------+------------------+-----------------------------+--------------------------------------------+--------+----------------------+
+---------------------+---------+--------------+
|      TIMESTAMP      |  ACTION | FLOATING IP  |
+---------------------+---------+--------------+
| 2017-07-13 15:00:57 | deleted | 10.8.180.199 |
+---------------------+---------+--------------+

+---------------------+---------+--------------------------------------+
|      TIMESTAMP      |  ACTION |              VOLUME ID               |
+---------------------+---------+--------------------------------------+
| 2017-07-13 15:00:57 | deleted | 358be8d1-6d4a-4db7-973f-8369d4ff86f7 |
+---------------------+---------+--------------------------------------+issuesRunning on CentOS7 even having EPEL and centos-release-openstack-newton I got this error in one of my CentOS
server.[root@centos7 yum.repos.d]# janitor
Traceback (most recent call last):
  File "/usr/bin/janitor", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3038, in <module>
    @_call_aside
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3022, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3051, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 659, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 672, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 862, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (requests 2.11.1 (/usr/lib/python2.7/site-packages), Requirement.parse('requests>=2.14.2'), set(['python-keystoneclient']))It is happening because python-requests lib is needed on version requests>=2.14.2 required by python-keystoneclient
when I ran:yum provides python-requeststhe latest version I got was 2.11 that still don't satisfacts.python-requests-2.6.0-1.el7_1.noarch : HTTP library, written in Python, for human beings
Repo        : base



python-requests-2.10.0-1.el7.noarch : HTTP library, written in Python, for human beings
Repo        : centos-openstack-newton



python2-requests-2.11.1-1.el7.noarch : HTTP library, written in Python, for human beings
Repo        : centos-openstack-newton
Matched from:
Provides    : python-requests = 2.11.1-1.el7



python2-requests-2.11.1-1.el7.noarch : HTTP library, written in Python, for human beings
Repo        : @centos-openstack-newton
Matched from:
Provides    : python-requests = 2.11.1-1.el7The trick here was, forcing reinstall requests using pip:pip install requests --upgrade