Package: hiproc
hiprocA library to help process HiRISE EDRs with ISIS.Free software: Apache 2 LicenseFeaturesTODO: Complete testing against Perl Pipelines.External DependenciesThese programs use as much ‘vanilla’ Python 3 as possible.However, it does depend on the following:pvl library (https://pvl.readthedocs.io)kalasiris library (https://kalasiris.readthedocs.io)gdalnumpyscipymatplotlibWarning !The algorithms based on the HiRISE Processing Pipelines were emulated
and tested locally, but the results of each pipelinehave notbeen tested directly against the results of the HiRISE Processing
Pipelines, and this warning will remain until I have done so.  As
a result, I would not particularly ‘trust’ anything produced by
these programs at this time, and consider these algorithms a
work-in-progress.These programs have been tested against their upstream Perl counterparts:EDR_Stats: Verified!Really just runshi2isisso no surprise here.HiCal: Verified.Upstream is undergoing change, needs to be re-verified once upstream
settles down.HiStitch: not verifiedHiccdStitch: not verifiedHiColorInit: not verifiedHiJitReg: not verifiedHiSlither: not verifiedHiColorNorm: not verifiedHiBeautify: not verifiedHiPrecisionInit: not verifiedHiNoProj: not verifiedHiJACK: not verifiedDocumentationFulldocumentation for hiproc is available,
including information on the processing flow of the various available programs, and
each program is self-documenting via their-hargument.Due to the interaction with ISIS and GDAL, please read the installation instructions
carefully.ContributingFeedback, issues, and contributions are always gratefully welcomed. See the
contributing guide for details on how to help and setup a development
environment.NamingThe ISIS software has a number of processing or “proc” programs
(mocproc,thmproc, etc.) that are meant to be run to process
raw images to higher-level, more usable versions.  Naming this
libraryhiprocis an echo to that. There is ahiprocprogram
that is available after installation that provides a streamlined
one-stop-program, but this package provides a great deal more.ChangelogAll notable changes to this project will be documented in this file.The format is based onKeep a Changelog,
and this project adheres toSemantic Versioning.When updating this file, please add an entry for your change underNot Yet Releasedand one of the following headings:Added - for new features.Changed - for changes in existing functionality.Deprecated - for soon-to-be removed features.Removed - for now removed features.Fixed - for any bug fixes.Security - in case of vulnerabilities.If the heading does not yet exist underNot Yet Released, then add it
as a 3rd level heading, underlined with pluses (see examples below).When preparing for a public release add a new 2nd level heading,
underlined with dashes underNot Yet Releasedwith the version number
and the release date, in year-month-day format (see examples below).Not Yet Released0.11.0 (2021-12-02)Addedfft_clean was added to the console programs list.  Still very experimental.ChangedUpdated EDR_Stats to be consistent with upstream EDR_Stats Pipeline version 3.0.0 (2021/09/08).
This handles the new LUT settings.Updated HiCal to be consistent with upstream HiCal Pipeline version 4.2.3 (2021/11/17).Updated HiStitch to be consistent with upstream HiStitch Pipeline version 2.21.2 (2021/09/09).
This change defaults to balance processing for all CCDs and corrects an indexing bug.FixedHiCal now deals with missing fields in the output of histats.lisfix now does not try to fit a slope to a column with masked values.bitflips now raises more descriptive exceptions when given poor data.0.10.0 (2021-08-05)Addedfft_clean functionality added for more testing, but not integrated with HiCal
or hiproc.FixedEDR_Stats was failing when some histats return values were None.  Now properly
just saves a “None” for SNR.HiCal multiprocessing approach was failing when a Channel was bypassed, essentially breaking the whole run.When the HiStitch function was called externally, a missing channel wasn’t handled properly. When called from the
HiStitch console script, it was fine.hiproc adjusted to take advantage of the above two fixes to HiCal and HiStitch.HiccdStitch now tolerant against the new ISIS stats return of the “N/A” string for missing values.0.9.0 (2021-07-08)AddedHiJACK now retains pre- and post-HiJACK flat files by default.HiJACK now has a –plotflats argument that can be used to plot the new flat filesaftera run of HiJACK.Added new rjplot program to plot the contents of the jitter and smear output files created
by resolve_jitter.FixedIn certain conditions an array can have zero length, and when it does, one shouldn’t try and
index it in a logging statement.0.8.1 (2021-07-06)AddedAdditional logging of the HiPrecisionInit output during a hiproc run, which was previously
obscured.FixedIn some legitimate cases, some elements returned from ISIS histat would be missing (e.g. in the
case of no valid pixels in the CAL_BUFFER area, there is no returned value of “Average”), but
were not handled properly (Issue #3).  More robust handling was added.0.8.0 (2021-06-30)Changedbitflips.py: Removed the span check when considering end values, which may affect how
the algorithm deals with “mostly good” DN histograms, such that it should now correctly
handle mesa-shaped DN histograms and not chop them in the middle.0.7.1 (2021-06-21)FixedThe new-in-0.7.0 multiprocessing requires that the multidict package be installed
(for pvl to use), and it wasn’t included in the requirements.  If you have 0.7.0,
a simple “conda install multidict” should get you working.0.7.0 (2021-06-15)AddedImproved documentationenvironment.yml for conda developmentAdded Python-based multiprocessing capabilities
to EDR_Stats, HiCal, and hiproc.ChangedIncorporated changes from upstream to HiCal to make bitflip and lisfix settings
on by default.Fixedresolve_jitter output header lines properly commented, now doesn’t bomb HiJACK’s run
of hijitter.now, additionally supporting ISIS 4.1.1 through 5.0.0Upstream addressed a bug in analyze_cubenorm_stats() that we had noticed,
so now fixed here, too.MANIFEST.in did not properly include the “data” directory, so that was a problem.0.6.1 (2021-03-23)Addedlisfix will return a non-zero exit code if it chooses not to fix the input cube.ChangedUpdated documentation in various places.0.6.0 (2021-03-18)AddedMore complete documentation for the programs and their parameters.ChangedImplemented better handling for configuration files so that they
will get distributed with the package.0.5.0 (2021-03-05)Addedlisfix: Added the lisfix module.bitflips: Sometimes the very end of the histogram (although not a formal minima) is the
appropriate choice, and is now considered.bitflips: Added capability to ignore minor maxima at the ends of the histogram to “roll down”
to a better solution.Changedname change of project from pyrise to hiproc.bitflips: Changed the default medstd_limit from 300 to 400 DN.Fixedbitflips: There were a variety of edge cases that resulted in errors.  The appropriate guardrails, handlers,
and recovery logic has now been added.HiStitch: The equalize and balance parameters cannot both be true.0.4.0 (2020-09-22)Tremendous amount of re-working in bitflips to improve
performance.Format cleanup0.3.0 (2020-05-16)Confirmed that EDR_Stats and HiCal produce identical output cubes.0.2.0 (2020-05-06)Updated with bit-flip correction.0.1.0 2020-03-21First shared on GitHub