Package: wardoff
WardoffWardoff (pronouncedward off) aim to help you to maintain your code base
clean and up-to-date by reducing the pain of collect informations about all
your underlaying libraries in your stack in a proactively manner.Wardoff grab for you many informations about your stacks, it can be used to:looking for deprecated stuffs in project requirements and underlyinggrab requirements' infos (detailed or not) (cf. see the examples in theinfossection)convert source code into python tokenFor each analyze a dedicated python virtual environment is built and project's
requirements are installed within. Then installed source code files of
project's requirement are analyzed one by one.Code analyze of the requirements is based onASTandpython tokens. Each
source code file of each underlaying library is analyzed in this way.You can pass a list of constraints to apply to your analyze to be sure
to match the right versions of your underlaying libraries.Traditionally projects maintainers are informed that functions will become
deprecated or removed by reading documentation or by
observing deprecation warning at the runtime in logs. When your stack
grow and the number of requirements in your stack increase it could be
painful to stay up-to-date, wardoff aim to collect for you all these infos
by using only 1 command without needing any runtime environment setup.InstallStill in development and really unstable, however you can install unstable
development versions by using:$python3-mpipinstall--userwardoffRequirementspython3.8+gitUsagesLooking for deprecated things in stacksThe main goal here is to analyze all requirements of a given project
to extract deprecated things from their codes.From a named packageFound deprecated things from a named package (directly from pypi):$wardoffniet# will list all deprecations founds in niet is requirements$wardoffoslo.messaging# will list all deprecations founds in oslo.messaging is requirements$wardoffoslo.messaging==12.2.2# will list all deprecations founds in oslo.messaging 12.2.2 is requirements$wardoffoslo.messaging==1.3.0# will list all deprecations founds in oslo.messaging 1.3.0 is requirementsFrom the current directoryRetrieve deprecated things from the current working directory.Retrieve requirements from:requirements.txttest-requirements.txt*-requirements.txt**/*requirements.txtExample:$wardoff# will list all deprecations founds in requirements founds in current directoryFrom a local repositoryRetrieve deprecated things from a distgit repo.Retrieve requirements from:requirements.txttest-requirements.txt*-requirements.txt**/*requirements.txtExample:$wardoff~/dev/nova# from a local clone of openstack/novaFrom a distant repository(Coming soon - not yet implemented)Retrieve deprecated things from a distgit repo.Example:$wardoffhttps://opendev.org/openstack/nova/# from opendev.org$wardoffhttps://github.com/openstack/nova# from github.com$wardoffgit@github.com:openstack/nova# by using git formattokenizerWardoff provide a CLI tokenizer which can be used against code passed through
the CLI or by passing a file path and a specific line to read.Example with raw code passed through the CLI:$wardoff-tokenizer"def person(name, age):"TokenInfo(type=62(ENCODING),string='utf-8',start=(0,0),end=(0,0),line='')TokenInfo(type=1(NAME),string='def',start=(1,0),end=(1,3),line='def person(name, age):')TokenInfo(type=1(NAME),string='person',start=(1,4),end=(1,10),line='def person(name, age):')TokenInfo(type=54(OP),string='(',start=(1,10),end=(1,11),line='def person(name, age):')TokenInfo(type=1(NAME),string='name',start=(1,11),end=(1,15),line='def person(name, age):')TokenInfo(type=54(OP),string=',',start=(1,15),end=(1,16),line='def person(name, age):')TokenInfo(type=1(NAME),string='age',start=(1,17),end=(1,20),line='def person(name, age):')TokenInfo(type=54(OP),string=')',start=(1,20),end=(1,21),line='def person(name, age):')TokenInfo(type=54(OP),string=':',start=(1,21),end=(1,22),line='def person(name, age):')TokenInfo(type=4(NEWLINE),string='',start=(1,22),end=(1,23),line='')TokenInfo(type=0(ENDMARKER),string='',start=(2,0),end=(2,0),line='')Another example by passing a file line to tokenize:wardoff-tokenizer~/dev/wardoff/wardoff/tokenizer.py+12It will tokenize the line number 12 of the file~/dev/wardoff/wardoff/tokenizer.pyFor further options with this command:$wardoff-tokenizer-hinfosWardoff allow you to retrieve infos of all your requirements. It will provide
a similar output thanpip showbut it will print all the requirements
related the given package. Returned output also contains the pypi classifiers
and also some useful infos that are not returned bypip show.
You can see that command as an advancedpip showwhere you can apply
different kinds of filters (cf. examples bellow)Examples:$wardoff-infososlo.cache
-----
name:certifi
version:2020.6.20
sources_path:/tmp/wardoff-25781/lib/python3.8/site-packages/certifi
summary:PythonpackageforprovidingMozilla'sCABundle.
home-page:https//certifiio.readthedocs.io/en/latest/
author:KennethReitz
author-email:me@kennethreitz.com
license:MPL-2.0
location:/tmp/wardoff-25781/lib/python3.8/site-packages
requires:
required-by:requests
DevelopmentStatus::5-Production/Stable
IntendedAudience::Developers
License::OSIApproved::MozillaPublicLicense2.0(MPL2.0)NaturalLanguage::English
ProgrammingLanguage::Python
ProgrammingLanguage::Python::3ProgrammingLanguage::Python::3.3
ProgrammingLanguage::Python::3.4
ProgrammingLanguage::Python::3.5
ProgrammingLanguage::Python::3.6
ProgrammingLanguage::Python::3.7
ProgrammingLanguage::Python::3.8
-----
name:chardet
version:3.0.4
sources_path:/tmp/wardoff-25781/lib/python3.8/site-packages/chardet
summary:UniversalencodingdetectorforPython2and3home-page:https//github.com/chardet/chardet
author:DanielBlanchard
author-email:dan.blanchard@gmail.com
license:LGPL
location:/tmp/wardoff-25781/lib/python3.8/site-packages
requires:
required-by:requests
DevelopmentStatus::4-Beta
IntendedAudience::Developers
License::OSIApproved::GNULibraryorLesserGeneralPublicLicense(LGPL)OperatingSystem::OSIndependent
ProgrammingLanguage::Python
ProgrammingLanguage::Python::2ProgrammingLanguage::Python::2.6
ProgrammingLanguage::Python::2.7
ProgrammingLanguage::Python::3ProgrammingLanguage::Python::3.3
ProgrammingLanguage::Python::3.4
ProgrammingLanguage::Python::3.5
ProgrammingLanguage::Python::3.6
Topic::SoftwareDevelopment::Libraries::PythonModules
Topic::TextProcessing::Linguistic
-----
...Previously, packages infos and classifiers have been displayed, in this way
by example it could allow you to find which package doesn't support specific
python version.Moreover you can usewardoff-infosto grab more informations than that,
by example you can retrieve all the projects home pages of your stack:$wardoff-infososlo.cache--keep-env--filterhome-page--no-separator--no-key
https//certifiio.readthedocs.io/en/latest/
https//github.com/chardet/chardet
https//docs.openstack.org/debtcollector/latest
https//github.com/micheles/decorator
https//github.com/sqlalchemy/dogpile.cache
https//github.com/kjd/idna
https//bitbucket.org/micktwomey/pyiso8601
https//msgpack.org/
https//github.com/drkjam/netaddr/
https//github.com/al45tair/netifaces
https//docs.openstack.org/oslo.config/latest/
https//docs.openstack.org/oslo.context/latest/
https//docs.openstack.org/oslo.i18n/latest
https//docs.openstack.org/oslo.log/latest
https//docs.openstack.org/oslo.serialization/latest/
https//docs.openstack.org/oslo.utils/latest/
https//github.com/pypa/packaging
https//docs.openstack.org/pbr/latest/
http//github.com/seb-m/pyinotify
https//github.com/pyparsing/pyparsing/
https//dateutil.readthedocs.io
http//pythonhosted.org/pytz
https//github.com/yaml/pyyaml
https//requests.readthedocs.io
http//rfc3986.readthedocs.io
https//github.com/benjaminp/six
https//docs.openstack.org/stevedore/latest/
https//urllib3.readthedocs.io/
https//github.com/GrahamDumpleton/wraptOr retrieve who are the main maintainers of the analyzed stack
(here in oslo.cache):$wardoff-infososlo.cache--filterauthor--no-separator--no-key|sort|uniq
AlastairHoughton
AndreyPetrov
BenjaminPeterson
DanielBlanchard
DavidP.D.Moss,StefanNordhausenetal
DonaldStufftandindividualcontributors
GrahamDumpleton
GustavoNiemeyer
IanStapletonCordasco
InadaNaoki
KennethReitz
KimDavies
KirillSimonov
MichaelTwomey
MicheleSimionato
MikeBayer
OpenStack
PaulMcGuire
SebastienMartini
StuartBishopAnother useful option is the--supportoption that allow you to retrieve
project that only support specific versions of python. You can mix supported
version by example to retrieve the project's requirements that support the
given versions, example:$wardoff-infososlo.cache--no-separator--support2.7,3.4--filtername--no-key
chardet
decorator
idna
netifaces
packaging
pyinotify
pyparsing
pytz
rfc3986
wraptThe previous example looking for requirements who still support python 2.7 and
3.4 in the same time, displayed results match this condition.Here is another example:$wardoff-infososlo.cache--no-separator--support3.7,3.8--filtername--no-key
certifi
idna
msgpack
netaddr
oslo.config
oslo.context
oslo.i18n
oslo.log
oslo.serialization
packaging
pyparsing
PyYAML
requests
urllib3
wraptThis kind of filter can be used to focus some of your developments or works
on specific versions of python.freezeWardoff allow you to freeze installed requirements. It will provide a similar
output thanpip freezebut it will only print requirements related the given
package.Example:$ wardoff-freeze oslo.messaging==1.3.0
amqp==2.6.0
Babel==2.8.0
certifi==2020.6.20
chardet==3.0.4
debtcollector==2.2.0
dnspython==2.0.0
eventlet==0.25.2
greenlet==0.4.16
idna==2.10
iso8601==0.1.12
kombu==4.6.11
monotonic==1.5
netaddr==0.8.0
oslo.config==8.3.1
oslo.i18n==5.0.0
pbr==5.4.5
pytz==2020.1
PyYAML==5.3.1
requests==2.24.0
rfc3986==1.4.0
six==1.15.0
stevedore==3.2.0
urllib3==1.25.10
vine==1.3.0
wrapt==1.12.1Wardoff's environmentTo works wardoff generate environment on your laptop, you can name
and reuse them. All the generated environment can be conserved by passing--keep-envor destroyed
(by default the current generated environment is destroyed at each execution).You can list your environments by usingwardoff-list-envor simply remove
all of them by using--wardoff-rm-env.If you pass a name at your command and if this environment already exist then
it will be reused, example:$wardoff-freeze~/dev/redhat/upstream/openstack/oslo/oslo.cache--ignore-extra-req--keep-env--envoslo-cache-vvv
INFO:Reusinganexistingenvironment/tmp/wardoff-oslo-cachecertifi==2020.6.20chardet==3.0.4debtcollector==2.2.0decorator==4.4.2
dogpile.cache==1.0.1idna==2.10iso8601==0.1.12msgpack==1.0.0netaddr==0.8.0netifaces==0.10.9
oslo.config==8.3.1
oslo.context==3.1.0
oslo.i18n==5.0.0
oslo.log==4.3.0
oslo.serialization==4.0.0
oslo.utils==4.4.0packaging==20.4pbr==5.4.5pyinotify==0.9.6pyparsing==2.4.7
python-dateutil==2.8.1pytz==2020.1PyYAML==5.3.1requests==2.24.0rfc3986==1.4.0six==1.15.0stevedore==3.2.0urllib3==1.25.10wrapt==1.12.1In the previous command we reused an existing environment calledoslo-cacheand we ask to keep this one at the end of our command to reusing it again. If
this environment doesn't exist it will created automatically.Output's VerbosityYou can increase the output verbosity on all your commands by passing the-voption. Thevyou pass the more the verbosity will be increased. By example-vvvwill display logs error, warn, info, and debug.This option could be useful to debug some situations and observe what's
happening.The future of wardoffWe plan to introduce more features like issues and pull requests or
patches harvesting.