Genre: Information/Explanation

Quality control guidelines The CCC grant involves transporting unique primary source materials between the Digital Production Centers of Duke , NCSU , and UNC-Chapel Hill for scanning . In the first few months of the grant , the Production & Materials Management Working Group developed guidelines for performing quality control at each Digital Production Center and workflows for verifying digital files . Below is an outline of the guidelines created for the grant . Duke scanning and postprocessing Scanning Following specific instructions developed for each collection , the scanner operator uses the appropriate equipment , settings and digitization guide to digitize the collection . Benchmark tests are performed and saved during each scanning session . During the capture process the operator monitors the images for quality and file naming errors . The images are saved in a structured way on the local drive and the digitization guide is updated to reflect the completion of an item . At the end of each shift , a DPC staff member moves the images to the DPC server . While the majority of the collection anomalies should be identified and resolved before digitization begins , if the scanner operator encounters an anomaly that is not covered in the scanning guidelines he or she stops digitization and reports the problem to the supervising DPC staff member . Quality Control 1 The Quality Control process is different depending on the scanning device and the nature of the material . Typically , unless automated during capture , the images are manually cropped ( leaving a 50 pixel background margin ) , de-skewed , correctly oriented , and processed at the proper resolution . The operator inspects all images for the correct file name , clipping , banding , blocking , color fidelity , uniform crop , and color profile . Quality Control 2 ( Derivative Creation ) Unless automated during capture , the operator manually crops images ( leaving no background ) and saves as JPEGs to a quality control folder on DPC server . During the second pass of quality control , student assistants inspect each image for : image consistency from operator to operator and image to image , assessing the cropping , dust removal , and border consistency , and perform any advanced processing needed to produce a quality digital surrogate . Finalize During this phase the DPC staff compares the digitization guide against the tiff and jpg files on the DPC server . Discrepancies such as missing files , misnamed files , and missing line items in the digitization guide are resolved . Once these issues are resolved display derivatives can be created using a script . NCSU scanning and postprocessing Scanning Scan materials using a BetterLight Super8KS at 300dpi ; save to TIFF , shooting vertical ( down ) . The scanning technician captures all of the source document with a minimal boundary so that all edges are visible in the resulting image . We can , but often do not , weigh down the edges or corners of the document . Quality control We view the image in Adobe PhotoShop at 100 % and pan around the four edges of the image looking for R , G , or B streaks that would indicate dust on the sensor . We pan through the interior of the image checking that the focus is good and there are no obvious errors or artifacts , physical or digital . We then check that color / appearance is approximate to that of the original and perform light touch-ups using unsharp masks to neaten the focus . We do not have to do this step in scans for other institutions if they would not like it to be performed . We usually do not scan oversized documents in an orientation that would require the rotation of the digital image , but all final images are in the proper orientation . Architectural drawings are usually scanned in the orientation of the title block , which can sometimes result in a sideways drawing . This is an editorial decision that can be adjusted . UNC scanning and postprocessing Scanning We scan materials on Zeutschels in TIFF format . When finalizing the scans from Zeutschels the files are named , moved to the appropriate drives , and the automated metadata is created by the scanning software and written to a text file . Next , we onvert TIFFs to JP2s in Adobe Bridge for long term storage . Finally , we create and OCR PDFs of each folder of images in Adobe Acrobat , and save the OCR as a txt file . Quality control When reviewing material , we 're mostly making sure the materials are readable . If documents are slightly skewed , slightly off color , etc. , we do not rescan them if the digital file is easily readable . This approach is specific to large-scale digitization and is taken with the idea that there is not enough time to review / fix every scan in a large scale digitization project . At the beginning of review for a batch of materials , we visually check every scan and compare to the analog materials . After the completion of the first couple of folders from a batch , we visually spot check the digital scans while comparing to the analog folders . Visual checks include : The filename assigned to the image is correct based on the statement of work ( UNC documentation for a batch of material to be scanned ) Color matches the original or is somehow " improved " The scan was done in the correct orientation Any image skew found is acceptable for the scan to be readable No dust , hair , specks of paper , etc are present Pages appear in the correct order for multi-page items Metadata recorded in the automated file from the Zeutschel is correct for the job / folder Deleting blank pages , and then renaming files after the deletion When creating PDFs , we scroll through PDFs looking for obvious errors . When uploading PDFs to ContentDM , we scroll through all of the scans to look for obvious errors . ContentDM will also deny the upload of files for many file name / resolution errors . When files are uploaded and linked to the online finding aid , we spot check images for quality issues , such as blurriness or distortion to make sure the upload was performed at " jp2 lossy ( 75 % ) " settings . Throughout all stages of quality control , we rescan if we find an error that makes the scans unreadable or distorted . If any institutions find errors with the scans produced by UNC for this grant , they can send the hollinger box back to the UNC DPC with a note on the tracking spreadsheet about what materials were missed / cut off / etc on the next delivery . The DPC will rescan the scan / folder accordingly , and send it back with the next batch of materials . Checksumming and digital file verification All three digital production centers are commited to implementing the BagIt specification and supporting tools for digital file packaging and transfer , checksum production , and checksum verification . This is a free tool that may require a few minutes of a technically-inclined person's time to install . BagIt packages collections of digital files into a " bag " that can include any number of subdirectories . A manifest file is created with the bag that lists all the checksums associated with the files within . See an example of the BagIt file structure below , using NCSU's file naming scheme . Every BagIt bag will be created BEFORE digital files are moved from the location to which they are saved during scanning . This enables checksum verification to catch any file corruption that might take place as files are moved around , for examples , as files are moved onto an external hard drive . While we originally planned for each BagIt bag to be created at the batch level ( with all materials sent in a single physical transfer to the scanning institution returned in a single bag ) , after the test run of the first BagIt transfer to NCSU from UNC in December 2011 we decided to create BagIt bags at the box level instead of the batch level . There are too many files in a bag for a desktop computer to have enough " umph " to validate with very large bags . The sending institution creates the bags , then moves them to an external hard drive . The sending institution then performs bag validation on the external hard drive , so that if something was corrupted in the move to the hard drive we do n't send over corrupt files . The sending institution SHOULD NOT look at any of the files on the hard drive with a Windows machine after bags have been created ; this seems to cause the creation of a thumbs . db file as soon as a network folder with graphics is opened in " filmstrip " or " thumbnail " mode in various Windows operating systems , which will later cause the bags not to validate . After files are delivered on external hard drives , home institutions have one week to verify manifest files for errors in checksum validation and to move the digital files to secure storage on their own servers . Home institutions should notify scanning institutions when they have validated files so that the scanning institutions can dump the files from their own drives . After one week , if scanning institutions have not heard from home institutions , they should call to verify that it 's ok to dump the files . There is no deadline for quality control of scanned images . If a home institution finds issues with scanned images after files have been dumped by the scanning institution , they will simply resend the hollingers with issues for rescanning along with a note on the transfer spreadsheet .