Package: drb
Data Request BrokerThis python module includes data model implementation of DRB (Data Request Broker).
Implementations are provided outside this project and shall be included into the application according to the needs.
Seedocumentationfor details.Library usageInstallingdrbwith execute the following in a terminal:pip install drbNotes for developersDev environment setupThe development environment includes all the features required to execute this module unitary test and to deploy it.
When loading the project with pycharm, the tool automatically reads requirements file and sets up the virtual environment.Installation of a virtual environment is not mandatory but greatly recommended in order to avoid any confusion between system or other projects libraries. Linux package python3-venv shall be installed (sudo apt install python3-venvon ubuntu)The command line to prepare the virtual environment:python3 -m venv venv
source venv/bin/activateOnce successfully run, the command line prompt is updated with the virtual environment name:(venv) $~> _Once the virtual environment installed, the dependencies shall be downloaded:(venv) $~> pip install -r requirements.txt --no-cache-dir
Collecting pip==21.1.2
  Downloading pip-21.1.2-py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 54 kB/s 
Collecting setuptools==57.0.0
  Downloading setuptools-57.0.0-py3-none-any.whl (821 kB)
     |████████████████████████████████| 821 kB 73 kB/s 
Installing collected packages: pip, setuptools
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
  Attempting uninstall: setuptools
    Found existing installation: setuptools 44.0.0
    Uninstalling setuptools-44.0.0:
      Successfully uninstalled setuptools-44.0.0
Successfully installed pip-21.1.2 setuptools-57.0.0At this stage the environment is installed.Other environment virtualization solution such asVagrantare also stable solutions for production.Git ContributionsContribution processWe are pleased to receive all users contributions. Each contribution shall be documented in english, and code styling shall followpep8recommendations.
You can also join the moderators team. Please contact me.Contribution process is based on gitlab best practice and processes the following schema:How to contributeContributor shall work in its own fork of the project and provides its contributions via the merge request process.
The merge request shall be approved and merged by a maintainers. So the step-by-step procedure to contribute is:Fork the project,Locally clone forked repository.Once created, the modification branch could be submitted as merge request inDraftproposal state.Once feature/bug finalized,Draftflag shall be removed and the assignees/maintainers notified for merge.Before submission, contributor shall clean up, squash and rebase its commits to be easily inserted as fast-forward into the main branch.
Please readhttps://chris.beams.io/posts/git-commitas commit writing best practices for this project.
The related issue/bug identifier shall be reported into the commit message, otherwise, the issue shall be commented/closed accordingly.Configuration managementThe project uses theversioneertool to manage releases deployement.
Then, the version management and deployment are coupled and can be performed in the same process.
This process secures the deployment process preventing a developer from accidentally deploying and erasing a release version. Versioneer tool checks the local repository then generates adirtyrelease when repository is not clean. This process can also be useful when developer needs to deploy snapshots.
Multiple and dirty deployement of versions are forbidden inpypi.orgrepository, this behavior also secures the version erasing risks.Setup the environmentThe environment shall be configured to deploy the python library onto Pypi public  repository.
The application used to manage module deployment istwine. This application shall be configured via${HOME}/.pypircfile as followed:Alternatively, the private gael's repository can be set (See[gael]entry.[distutils]index-servers=pypidrb[pypi]username:__token__password:pypi-XXX[drb]repository=https://upload.pypi.org/legacy/username:__token__password:pypi-YYY[gael]repository:https://repository.gael-systems.com/repository/python-gael/username:usernamepassword:passwordThe important part is thedrbsection which defines the remote repository and credentials or token for the deployment (Seehttps://pypi.org/help/#apitokenfor details).Perform the release and deploymentThe version management is performed automatically with git tags. Setting the version is coupled with the deployment process within the CI/CD process.To generate a new version, tag the master branch with the expected version and push the new tag version into git:git tag 1.0-rc1
git push origin 1.0-rc1On pushing new tag, a pipeline is automatically executed to controlcode format compliance withpep8)code source security with pluginsbanditandsemgrepcode unittary tests with python 3.8 and python 3.9code coverage computationdeploy the release into pypi repository.NB: if the runner is not properly configured to upload distribution to the remote repository, the pipeline executed at tag push time might fail. To perform the distribution manually it shall be possible to run the following command lines:make dist-clean
make dist-deploySee the makefile details in command line sectionhere after.Branching modelIn git, thanks to the tag oriented release, the branching model fixes the following rules:All the contributions are merged into themainbranch.Contribution Merge Requests are merged using fast-forward mode.A dedicatedrelease-xxbranch can be created for post-release fixes(hotfixes).Project Final Releases StategyThe project release version is based on 3 digits "M.m.f" :M: Major version is created when DRB API is modified or interface behavior changed. (i.e. index are used instead of occurrence does not change interface, but greatly change the code behavior) - Usually Major version is created when unitary tests are impacted when they concerns module interface validation.m: Minor version is created when new features are introduced without impact on the previous interface: Adding new entries in classes, new classes, add feature in interface method or new feature. (i.e. get_impl is able to manage kwargs dictionary argument to be passed to the sub implementation). Usually minor version is created when unitary tests are added to validate new feature, but old ones stays unchanged.f: Fix version is created when a fix or a set of fixes are released. New unitary tests shall be created to ensure non regression of the issue in future changes.Tag management and pre/post releasesDrb project follows thepep440recommendations for the tag representation. Versions are represented with the couple of (Major, Minor) version numbers for all pre-releases. Modifiers such as alpha (aN), beta(bN), release candidate(rcN), or post release (.postN) are possible as followed:alpha version:1.0a1beta version:1.0b1release candidate version:1.0rc1Final release:1.0.0Note that (a,b,rc) are based on 2 digits (Major.Minor) whereas Final release are identified on 3 digits as desibed in theprevious chapter: The third digit is used to manage fixes, waiting minor releases.Command lineThe environment comes with a preconfigured Makefile able to set up and prepare python environment to run tests and coverage. It also provides target to deploy new release manually.make cleanClean-up the environment from cache and lightweight components. It does not removed downloaded dependencies (from venv directory), nor distributions.make dist-cleanThedist-cleancommand full cleans the repository as it has been cloned first.
Following the call ofdist-cleanthe virtual environment and all the caches will be removed.make testRun the unitary tests.make lintCheck if the source code properly follows the pep8 directives. This test is also used in gitlab piplines to accept pushed sources.make coverageRun the test coverage and generates a html report intohtmlcovdirectory.make distPrepare a distribution locally intodistdirectory. When no tag is present on the current commit, or modified files are present into the local directory, the distribution process creates a dirty version to be safety deploy to the repository.make dist-deployPrepare and deploy a distribution into the gael's remote Pipy repository.
This command is run automatically when pushing a new tag.LimitationCurrently, DRB project is not tested on Windows environment and some issues may appear on Windows systems.If any troubleshoot occurred with the DRB library, please report us, opening a new issue or accident on the project GitLab page.