Package: jprofile
JprofileWhat isjprofile?Jprofileis a simple tool for automated profiling of large batches ofJP2(JPEG 2000 part 1) images. Internally it wraps aroundjpylyzer, which is used for validating each image and for extracting its properties. Thejpylyzeroutput is then validated against a set ofSchematronschemas that contain the required characteristics for master, access and target images, respectively.InstallationThe easiest method to installJprofileis to use thepippackage manager. Alternatively, Windows users can also use stand-alone binaries that don't require Python at all (see below).Installation with pip (single user)This will work on any platform for which Python is available. You need a recent version ofpip(version 9.0 or more recent). To installJprofilefor a single user, use the following command:pip install jprofile --userInstallation with pip (all users)To installJprofileforallusers, use the following command:pip install jprofileYou need local admin (Windows) / superuser (Linux) privilige to do this. On Windows, you can do this by running the above command in a Command Prompt window that was opened as Administrator. On Linux, use this:sudo pip install jprofileInstallation of Windows binariesFor Windows users who don't have Python available on their system, stand-alone binaries ofJprofileare available. In this case the installation steps are:Download the latest binaries (64 or 32 bit) from thelatest release page.Unzip the downloaded file to an empty directory.Command-line syntaxusage: jprofile batchDir prefixOut -p PROFILEPositional argumentsbatchDir: root directory of batchprefixOut: prefix that is used for writing output filesPROFILE: name of profile that defines schemas for master, access and target imagesTo list all available profiles, use a value oflorlistforPROFILE.Batch structureJprofilewas designed for processing digitisation batches that are delivered to the KB by external suppliers. These batches typically contain (losslessly compressed)masterJP2s, (lossily compressed)accessJP2s and (sometimes) alsotechnical targetJP2s. These are located in a folder structure that contains (sub) directories namedmaster,accessandtargets-jp2, respectively. Below is an example:./testbatch
├── access
│   ├── IMAGE000060.jp2
│   ├── IMAGE000061.jp2
│   ├── IMAGE000062.jp2
│   ├── ::
│   ├── IMAGE000080.jp2
│   └── IMAGE000081.jp2
├── master
│   ├── IMAGE000060.jp2
│   ├── IMAGE000061.jp2
│   ├── IMAGE000062.jp2
│   ├── ::
│   ├── IMAGE000080.jp2
│   └── IMAGE000081.jp2
└── targets-jp2As long as a batch follows this basic structure,Jprofilecan handle it. Note that:master,accessandtargets-jp2directories may occur at different nesting levels. This is no problem, sinceprofilerecursively traverses all subdirctories in a batch.if either amaster,accessortargets-jp2directory is missing,jprofilewill simply ignore it (i.e. it's perfectly OK if your batch only containsmasterimages).Batches may contain other folders. These are ignored byjprofile.ProfilesAprofileis anXML-formatted file that simply defines which schemas are used to validatejpylyzer's output for master, access and target images, respectively. Here's an example:<profile>

<!-- Sample profile -->

<schemaMaster>master300Gray_2014.sch</schemaMaster>
<schemaAccess>access300Colour_2014.sch</schemaAccess>
<schemaTarget>master300Colour_2014.sch</schemaTarget>

</profile>Note that each entry only contains thenameof a profile, not its full path! All profiles are located in theprofilesdirectory in the installation folder.Available profilesThe following profiles are included by default:NameDescriptionkb_generic_2014.xmlGeneric profile for KB digitisation streams (doesn't include any checks on resolution or colour spaces!)kb_300Colour_2014.xmlAs generic profile, but with additional requirements on resolution (must be equal to 300 ppi) and colour space (must be Adobe RGB 1998)kb_300Gray_2014.xmlAs generic profile, but with additional requirements on resolution (must be equal to 300 ppi) and colour space (must be Gray Gamma 2.2)kb_600Colour_2014.xmlAs generic profile, but with additional requirements on resolution (must be equal to 600 ppi) and colour space (must be Adobe RGB 1998)kb_600Gray_2014.xmlAs generic profile, but with additional requirements on resolution (must be equal to 600 ppi) and colour space (must be Gray Gamma 2.2)It is possible to create custom-made profiles. Just add them to theprofilesdirectory in the installation folder.SchemasThe quality assessment is based on a number of rules/tests that are defined a set ofSchematronschemas. These are located in theschemasfolder in the installation directory. In principleanyproperty that is reported byjpylyzercan be used here, and new tests can be added by editing the schemas. More details on this can be found inthis blog post.Available schemasNameDescriptionkbMaster_2014.schGeneric schema for losslessly-compressed master images according to 2014 specificationsmaster600Colour_2014.schSchema for losslessly-compressed master images, 600 ppi, Adobe RGB (1998) colour spacemaster600Gray_2014.schSchema for losslessly-compressed master images, 600 ppi, Gray Gamma 2.2 colour spacemaster300Colour_2014.schSchema for losslessly-compressed master images, 300 ppi, Adobe RGB (1998) colour spacemaster300Gray_2014.schSchema for losslessly-compressed master images, 300 ppi, Gray Gamma 2.2 colour spacekbAccess_2014.schGeneric schema for lossily-compressed access images according to 2014 specificationsaccess600Colour_2014.schSchema for lossily-compressed access images, 600 ppi, Adobe RGB (1998) colour spaceaccess600Gray_2014.schSchema for lossily-compressed access images, 600 ppi, Gray Gamma 2.2 colour spaceaccess300Colour_2014.schSchema for lossily-compressed access images, 300 ppi, Adobe RGB (1998) colour spaceaccess300Gray_2014.schSchema for lossily-compressed access images, 300 ppi, Gray Gamma 2.2 colour spaceIt is possible to create custom-made schemas. Just add them to theschemasdirectory in the installation folder.Overview of 2014 schemasThe following tables give a general overview of the technical profiles that the generic master- and access schemas are representing:MasterParameterValueFile formatJP2 (JPEG 2000 Part 1)Compression typeReversible 5-3 wavelet filterColour transformYes (only for colour images)Number of decomposition levels5Progression orderRPCLTile size1024 x 1024Code block size64 x 64 (26x 26)Precinct size256 x 256 (28) for 2 highest resolution levels; 128 x 128 (27) for remaining resolution levelsNumber of quality layers11Target compression ratio layers2560:1 [1] ; 1280:1 [2] ;  640:1 [3] ; 320:1 [4] ; 160:1 [5] ; 80:1 [6] ; 40:1 [7] ; 20:1 [8] ; 10:1 [9] ; 5:1 [10] ; 2.5:1 [11]Error resilienceStart-of-packet headers; end-of-packet headers; segmentation symbolsSampling rateStored in "Capture Resolution" fieldsCapture metadataEmbedded as XMP metadata in XML boxAccessParameterValueFile formatJP2 (JPEG 2000 Part 1)Compression typeIrreversible 7-9 wavelet filterColour transformYes (only for colour images)Number of decomposition levels5Progression orderRPCLTile size1024 x 1024Code block size64 x 64 (26x 26)Precinct size256 x 256 (28) for 2 highest resolution levels; 128 x 128 (27) for remaining resolution levelsNumber of quality layers8Target compression ratio layers2560:1 [1] ; 1280:1 [2] ;  640:1 [3] ; 320:1 [4] ; 160:1 [5] ; 80:1 [6] ; 40:1 [7] ; 20:1 [8]Error resilienceStart-of-packet headers; end-of-packet headers; segmentation symbolsSampling rateStored in "Capture Resolution" fieldsCapture metadataEmbedded as XMP metadata in XML boxNote that jpylyzer is unable to establish the compression ratio of individual layers, so the access schema only checks for the overall compression ratio (i.e. 20:1). The more specific schemas (300Colour, 600Gray, etc.) contain additional checks for resolution values, the number of colour components and embedded ICC profiles.Usage examplesList available profilesjprofile d:\myBatch mybatch -p listThis results in a list of all available profiles (these are stored in the installation folder'sprofilesdirectory):Available profiles:

kb_600Gray_2014.xml
kb_300Gray_2014.xml
kb_300Colour_2014.xml
kb_600Colour_2014.xml
kb_generic_2014.xmlAnalyse batchjprofile d:\myBatch mybatch -p kb_300Colour_2014.xmlThis will result in the creation of 2 output files:mybatch_status.csv(status output file)mybatch_failed.txt(detailed output on images that failed quality asessment)Status output fileThis is a comma-separated file with the assessment status of each analysed image. The assessment status is eitherpass(passed all tests) orfail(failed one or more tests). Here's an example:F:\test\access\MMKB03_000004896_00015_access.jp2,pass
F:\test\access\MMKB03_000004896_00115_access.jp2,pass
F:\test\access\MMKB03_000004896_00215_access.jp2,pass
F:\test\targets-jp2\MMKB03_MTF_RGB_20120626_02_01.jp2,fail
F:\test\master\MMKB03_000004896_00015_master.jp2,passFailure output fileAny image that failed one or more tests are reported in the failure output file. For each failed image, it contains a full reference to the file path, followed by the specific errors. An example:F:\test\targets-jp2\MMKB03_MTF_RGB_20120626_02_01.jp2
*** Schema validation errors:
Test "layers = '11'" failed (wrong number of layers)
Test "transformation = '5-3 reversible'" failed (wrong transformation)
Test "comment = 'KB_MASTER_LOSSLESS_01/01/2015'" failed (wrong codestream comment string)
####Entries in this file are separated by a sequence of 4 '#' characters. Note that each line here corresponds to a failed test in the schema. For images that are identified as not-valid JP2 some additional information fromjpylyzer's output is included as well. For example:F:\test\master\MMUBL07_MTF_GRAY_20121213_01_05.jp2
*** Schema validation errors:
Test "isValidJP2 = 'True'" failed (no valid JP2)
*** Jpylyzer JP2 validation errors:
Test methIsValid failed
Test precIsValid failed
Test approxIsValid failed
Test foundNextTilePartOrEOC failed
Test foundEOCMarker failed
####Here, the outcome of testisValidJP2means that the image does not conform to theJP2specification. The lines following 'Jpylyzer JP2 validation errors' lists the specific errors that were reported byjpylyzer. The meaning of these errors can be found in thejpylyzerUser Manual.PreconditionsAll images that are to be analysed have a .jp2 extension (all others are ignored!)Masterimages are located in a (subdirectory of a) directory called 'master'Accessimages are located in a (subdirectory of a) directory called 'access'Targetimages are located in a (subdirectory of a) directory called 'targets-jp2'.Either of the above directories may be missing.Other than that, the organisation of images may follow any arbitrary directory structure (jprofiledoes a recursive scan of whole directory tree of a batch).Known limitationsImages that have names containing square brackets ("[" and "]" are ignored (limitation ofPython'sglobmodule, will be solved in the future).LicensingJprofileis released under theGNU Lesser General Public License.Useful linksjpylyzerSchematronAutomated assesment of JP2 against a technical profile using jpylyzer and Schematron