Package: dkist-processing-visp
OverviewThe dkist-processing-visp library contains the implementation of the visp pipelines as a collection of thedkist-processing-coreframework anddkist-processing-commonTasks.The recommended project structure is to separate tasks and workflows into seperate packages.  Having the workflows
in their own package facilitates using the build_utils to test the integrity of those workflows in the unit test.Calibration PipelineBuildArtifacts are built throughbitbucket pipelines.The pipeline can be used in other repos with a modification of the package and artifact locations
to use the names relevant to the target repo.e.g. dkist-processing-test -> dkist-processing-vbi and dkist_processing_test -> dkist_processing_vbiDeploymentDeployment is done withturtlebotand follows
the process detailed indkist-processing-coreEnvironment VariablesOnly those specified bydkist-processing-coreanddkist-processing-common.Developmentgitclonegit@bitbucket.org:dkistdc/dkist-processing-visp.gitcddkist-processing-visppre-commitinstallpipinstall-e.[test]pytest-v--covdkist_processing_vispChangelogWhen you makeanychange to this repository itMUSTbe accompanied by a changelog file.
The changelog for this repository uses thetowncrierpackage.
Entries in the changelog for the next release are added as individual files (one per change) to thechangelog/directory.Writing a Changelog EntryA changelog entry accompanying a change should be added to thechangelog/directory.
The name of a file in this directory follows a specific template:<PULL REQUEST NUMBER>.<TYPE>[.<COUNTER>].rstThe fields have the following meanings:<PULL REQUEST NUMBER>: This is the number of the pull request, so people can jump from the changelog entry to the diff on BitBucket.<TYPE>: This is the type of the change and must be one of the values described below.<COUNTER>: This is an optional field, if you make more than one change of the same type you can append a counter to the subsequent changes, i.e.100.bugfix.rstand100.bugfix.1.rstfor two bugfix changes in the same PR.The list of possible types is defined the the towncrier section ofpyproject.toml, the types are:feature: This change is a new code feature.bugfix: This is a change which fixes a bug.doc: A documentation change.removal: A deprecation or removal of public API.misc: Any small change which doesn’t fit anywhere else, such as a change to the package infrastructure.Rendering the Changelog at Release TimeWhen you are about to tag a release first you must runtowncrierto render the changelog.
The steps for this are as follows:Runtowncrier build –version vx.y.zusing the version number you want to tag.Agree to have towncrier remove the fragments.Add and commit your changes.Tag the release.NOTE:If you forget to add a Changelog entry to a tagged release (either manually or automatically withtowncrier)
then the Bitbucket pipeline will fail. To be able to use the same tag you must delete it locally and on the remote branch:# First, actually update the CHANGELOG and commit the updategitcommit# Delete tagsgittag-dvWHATEVER.THE.VERSIONgitpush--deleteoriginvWHATEVER.THE.VERSION# Re-tag with the same versiongittagvWHATEVER.THE.VERSIONgitpush--tagsoriginmainScience ChangelogWhenever a release involves changes to the scientific quality of L1 data, additional changelog fragment(s) should be
created. These fragments are intended to be as verbose as is needed to accurately capture the scope of the change(s),
so feel free to use all the fancy RST you want. Science fragments are placed in the samechangelog/directory
as other fragments, but are always called:<PR NUMBER | +>.science[.<COUNTER>].rstIn the case that a single pull request encapsulates the entirety of the scientific change then the first field should
be that PR number (same as the normal CHANGELOG). If, however, there is not a simple mapping from a single PR to a scienctific
change then use the character “+” instead; this will create a changelog entry with no associated PR. For example:$lschangelog/99.bugfix.rst# This is a normal changelog fragment associated with a bugfix in PR 9999.science.rst# Apparently that bugfix also changed the scientific results, so that PR also gets a science fragment+.science.rst# This fragment is not associated with a PRWhen it comes time to build the SCIENCE_CHANGELOG, use thescience_towncrier.shscript in this repo to do so.
This script accepts all the same arguments as the defaulttowncrier. For exmaple:./science_towncrier.shbuild--versionvx.y.zThis will update the SCIENCE_CHANGELOG and remove any science fragments from the changelog directory.