Package: python-package-sync-tool
PythonPackageSyncToolPythonPackageSyncTool is a Python utility to "fix" requirements.txt.It is used to make manipulation on requirements.txt that is produces bypip freezeGetting HelpQuickStartFirst of all it is better to installalex_ber_utils(this dependency is used insetup.py).pip3install-Ualex_ber_utilsThan:pip3install-Upython-package-sync-toolIn some directory createconfig.yml. Here is example
(https://github.com/alex-ber/alpine-anaconda3/blob/master/python_package_sync_tool_config.yml)treeroot:source:installed.txtdestination:rm.txtmutual_exclusion:Trueremove:-conda-conda-package-handling-icc-rt-intel-openmp-mkl-mkl-fft-mkl-random-mkl-service-pycosat-sip-tbbadd:#    - graphviz==2.38.0#    - entrypoints==0.2.3#    - numpy==1.16.2#    - PyYAML==5.1#    - sasl==0.2.1#    - scipy==1.2.1#    - tables==3.4.4#    - terminado==0.8.2#    - tornado==5.1.1#    - Twisted==19.2.0Createinstalled.txtwith installed packages (inpip freezeformat).Now, you can runpython3-malexber.reqsync.dataYou can also override parameters in the yaml file:python3-malexber.reqsync.data--treeroot.add=some_new_package:1.0.0shorthandpython3-malexber.reqsync.data--add=some_new_package:1.0.0is also supported.Of course, you can provide all parameters via CLI and not to use YAML file.You can also supply--general.log(no shorthand here) parameter (you can do it also viaconfig.yml).
There is some default log configuration (to write everything tostdout).Note:config.ymlnow also supports format thatalex-ber-utilshttps://github.com/alex-ber/AlexBerUtilssupport for a long time.Please, consulthttps://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998cfor
full supported list ofgeneralparameters.See documentation hereMyparsermodulehttps://medium.com/analytics-vidhya/my-parser-module-429ed1457718Myymlparsersmodulehttps://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6My majorinit_app_confmodulehttps://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998cNote:Semicolomn and not equal sign is used here due to Python limitaion of usage of equal sign in the value in the command line.You can specified multiple packages using comma delimiter.You can specify path to your config file using--config_file(or--general.config_file).
It can be absolute or relative.Alternatively,cd/opt/anaconda3/lib/python3.8/site-packages/alexber/reqsync/data/Note: This is Path where you're actually install my utility, it can be different in your machine.If you use venv it will look something like:cd/opt/MyProject/venv/Lib/site-packages/alexber/reqsyncAlternatively you can create script file for yourself, named, say, driver.py:#!/usr/bin/python3importalexber.reqsync.appasappif__name__=="__main__":app.main()Than create file config.yml near your script (see data/config.yml) or provide all parameter using command line
argruments. Use ':' in places where you should naturally write '==' (see explanation below).Parameters 'source' and 'destination' are required. You should also provide (requirements) file for 'source'.mutual_exclusionhas default value True.Now, typepython3-malexber.reqsync.data--add=some_new_package:1.0.0or if you're using script (driver.py) go the directory with the script and type./driver.py--add=some_new_package:1.0.0or if you install my tool to Anaconda/Python/venv that has it's bin folder is in the Path
you can runpython_package_sync_tool--add=some_new_package:1.0.0or alternativleyyou can runreqsync--add=some_new_package:1.0.0This will run quick check whether package is not in remove list. If it is, the utility will fail. You can override this
beahivor by supplying--mutual_exclusion=False.Then, this will add some_new_package with version 1.0.0 to the requirements-dest.txtNote:Semicolomn and not equal sign is used here due to Python limitaion of usage of equal sign in the value in the command line.You can specified multiple packages using comma delimiter.You can specifiy path to your config file using--config_file(or--general.config_file).It can be absolute or relative. If you're running using script (driver.py), that it can be relative to the directory
whether you put your script. If you're running as the module (python3 -m), it can be relative to/opt/anaconda3/lib/python3.8/site-packages/alexber/reqsync/data/(exact path can be different, see above).You can supplied multiply packages by using comma:python3-malexber.reqsync.data--add=some_new_package:1.0.0,another_new_package:2.0.0or if you're using script (driver.py) go the directory with the script and type./driver.py--add=some_new_package:1.0.0,another_new_package:2.0.0Installing from Githubpython3-mpipinstall-Uhttps://github.com/alex-ber/PythonPackageSyncTool/archive/master.zipOptionally installing tests requirements.python3-mpipinstall-Uhttps://github.com/alex-ber/PythonPackageSyncTool/archive/master.zip#egg=python-package-sync-tool[tests]Or explicitly:wgethttps://github.com/alex-ber/PythonPackageSyncTool/archive/master.zip-Omaster.zip;unzipmaster.zip;rmmaster.zipAnd then installing from source (see below).Installing from sourcepython3-mpipinstall.# only installs "required"python3-mpipinstall.[test]# installs dependencies for testsUsing Dockeralexberkovich/python_package_sync_tool:latestcontains allpython_package_sync_tooldependencies.
This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile.Alternatively, you can use it as base Docker image for your project and add/upgrade
another dependencies as you need.For example:FROMalexberkovich/python_package_sync_tool:latestCOPYrequirements.txtetc/requirements.txtRUNset-ex&&\#latestpip,setuptools,wheelpipinstall--upgradepipsetuptoolswheel&&\pipinstallpython_package_sync_toolpipinstall-retc/requirements.txtCMD["/bin/sh"]#CMD tail -f /dev/nullwhererequirements.txtis requirements for your project.From the directory with setup.pypython3setup.pytest#run all testspytestRequirementsPythonPackageSyncTool requires the following modules.Python 3.7+alex-ber-utils==0.6.2alex-ber-utils[yml]==0.6.2Seehttps://github.com/alex-ber/AlexBerUtilsChangelogAll notable changes to this project will be documented in this file.#https://pypi.org/manage/project/python-package-sync-tool/releases/[0.5.3] - 2020-12-17ChangedREAMDE.md, requiremetes section.[0.5.2] - 2020-12-17Removedalexber.reqsync.utils.app_confremoved.ChangedPytest version updated (also some tests dependencies changed)alex-ber-utils dependency updated.Logger information was moved fromapp.pytoconfig.ymlfile.config.ymlnow also supports format thatalex-ber-utilshttps://github.com/alex-ber/AlexBerUtilssupport for a long time.Please, consulthttps://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998cfor
full supported list ofgeneralparameters.See documentation hereMyparsermodulehttps://medium.com/analytics-vidhya/my-parser-module-429ed1457718Myymlparsersmodulehttps://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6My majorinit_app_confmodulehttps://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998cLogger is enabled, relative path is resolved correctly, and other little tweaks, seeIntegrating Python’s logging and warnings packages
[https://medium.com/analytics-vidhya/integrating-pythons-logging-and-warnings-packages-7ffd6f65e02d]Making relative path to file to work
[https://alex-ber.medium.com/making-relative-path-to-file-to-work-d5d0f1da67bf]config.xml- to remove package was added mkl*-related packagages and anaconda-specific packages.REAMDE.mdwas updated to reflect all changes above.AddedOptional Docker support added.[0.5.1] - 2020-10-25AddedSupport for Python 3.8Changed[0.4.3] - 2019-10-17Changedanaconda-navigator and conda-build added to config.yml.[0.4.2] - 2019-10-16Changednavigator-updater added to config.yml.[0.4.1] - 2019-05-30ChangedBug fix: adding packages before all existing one works incorrect.Removing alexber.reqsync.utils.parsers. It was fully duplicated by alexber.utils.parsers.
So, all usage was change to the latest (part of alex-ber-utils).[0.3.1] - 2019-05-23ChangedBug fix: adding packages before all existing one works incorrect.AddedUnit test for bug fix that check adding packages before all existing one.More detail assertion to integration tests.Unit-test for non-sorted requirements-src.Unit-test that check run with empty add and empty remvoe.Unit-test that check removing single package.Unit-test that check remove first package in requirements-src.Unit-test that check remove last package in requirements-src.Unit-test that check that empty lines in requirements-src are ignored.Unit-test that check correct usage of file input buffer and file output buffer.[0.2.11] - 2019-05-22ChangedDependency alex-ber-utils bumped up to 0.2.5.[0.2.8] - 2019-05-22ChangedDependency alex-ber-utils bumped up to 0.2.4.[0.2.6] - 2019-05-22ChangedDependency alex-ber-utils bumped up to 0.2.3.[0.2.5] - 2019-05-22ChangedFixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload)Dependency alex-ber-utils bumped up to 0.2.2.[0.2.4] - 2019-05-22ChangedAdding dependency alex-ber-utils 0.2.1 to README.md.[0.2.3] - 2019-05-22ChangedUpgrading urllib3, SQLAlchemy, pycrypto dependenies beacause of volnurabilities issues.[0.2.2] - 2019-05-22ChangedFixing python-package-sync-tool.Creating alias reqsync to python-package-sync-tool.Some minour fixed.[0.2.1] - 2019-05-22ChangedChanging dependency version of alex-ber-utils to 0.2.1.[0.2.0] - 2019-05-22ChangedOnly bumping up version.[0.1.9] - 2019-05-22ChangedAdded alex-ber-utils as dependency.Deleting old README-old.rst file.requirements-src.txt updated.Clarification added to README.md that alex_ber_utils should be installed first.formatting CHANGELOG.MD (minor fix)Added alternative of usage of python_package_sync_tool to README.mdFixing bug that --add is emptyFactor out tests_data to seperate folder, use importlib.resources API.Updated README-old.rst[0.1.8] - 2019-05-20ChangedREADME.md change, key '-U' added to pip3 install.[0.1.7] - 2019-05-20RemovedSome project cleanup.ChangedCHANGELOG and REAMDE now use Markdown format.REAMDE totally rewritten.Fixing bugs in the core algorithm. Simplifying code.Fixing correct handling of package adding to the buttom of the list.[0.1.6] - 2019-05-20Added__init__.pyfile added to alexber.reqsync.data.[0.1.5] - 2019-05-20AddedSmall tool to sync package from different machines.