DOI: 10.5061/dryad.v15dv4272
title: Data from: High-throughput classification and quantification of skinning phenotype in sweet potatoes

description: Sweetpotatoes (SPs) (Ipomoea batatas) are crops valued for their color,
 flavor, and nutrition.  Harvesting is labor-intensive, requiring
 hand-picking to maintain skin quality.  Mechanical harvesting
 often causes skin damage, known as “skinning,” where skin is cut, scraped,
 or torn, leading to lower quality during packing. To manage this, packers
 may conduct a costly “field-switch” to reduce skinning in the production
 line. Currently, skinning levels are visually assessed by the packers and
 stakeholders.  Field-switches involve transitioning between
 multiple fields during harvest to meet specific customer orders (e.g.,
 supermarkets, processing plants, or end users) that require higher-quality
 SPs. This process aims to minimize skinning and ensure the SPs meet the
 desired quality standards for those orders.  This study
 introduces a computer vision (CV) pipeline to automate skinning assessment
 using a ResNet50-based DeepLabV3+ semantic segmentation model. The CV
 system was trained to identify three classes: skinning, intact
 skin\textbackslash SP, and background. A machine vision camera, mounted
 above a conveyor belt, captured images throughout several full production
 days. The pipeline calculated the percentage of skinning PS as the ratio
 between the predicted skinning area and the total SP surface area (As +
 Ap). Using this method, daily production trends and field-switch decisions
 were studied with image data from six production days, chosen by our
 grower collaborator. Percent skinning ratings calculated from random
 sub-samples of imaged SPs—where only a portion of the full image set was
 analyzed—showed no significant differences compared to those derived from
 complete image sets. This demonstrates that sub-sampling can reduce
 computational processing times by 90% while maintaining accuracy. When
 data was binned in 30-minute intervals, field-switches occurred when there
 was approximately 1.5 PS. Across 2,417,907 SP instances, the model
 achieved a root mean square error (RMSE) of 0.55%, R^2 of 0.84, 80.03%
 recall, 99.99% specificity, 77.44% F1 score, and 99.98% grading accuracy.
 This offers a promising improvement for automatic skinning detection on a
 commercial scale.
description: <strong>Data Collection</strong>
 The data collection process involved reviewing millions of SPs
 over a two-year period, from late winter 2022 to late winter 2024. Based
 on discussions with our industry collaborator, we identified specific days
 when skinning issues were most apparent. SPs for the training dataset were
 chosen for their fresh skinning marks, which were likely caused by
 harvesting, rough handling, improper curing, washing, or damage from
 machinery. These skinning issues typically occurred within one to two
 weeks of the harvest, guiding the selection of images.
 The images collected were stored in .jpg format with a ground
 sample distance (GSD) of 0.521 mm/pixel. Most images had a size of
 866x1599 pixels, though a few were smaller and resized to 866x1599 pixels
 using MATLAB. The data collection process was ongoing throughout the
 study, with adjustments made to the dataset after each performance
 evaluation to ensure that only the most relevant and high-quality images
 were retained. <strong>Data Curation and
 Labeling</strong> A dataset of individual SP
 instances was curated by selecting high-quality images with distinct
 skinning marks. These images were chosen based on input from our industry
 collaborator, who helped identify periods when skinning issues were most
 prevalent. The curated dataset consisted of high-quality images from five
 distinct days spanning late winter 2022 to late winter 2024. The SPs were
 selected based on their fresh skinning marks, which were presumed to have
 occurred within the past one to two weeks due to factors such as
 harvesting, rough handling, improper curing, washing, and mechanical
 damage (bumps, scrapes, or falls). Each image was
 manually annotated with three categories: skinning, SP, and background.
 The skinning marks ranged in size from 0.36 mm² to 316.17 mm². The images
 were taken with a machine vision camera at the eliminator table. Once
 annotated, the images were processed further for model training. To
 augment the dataset, each image was randomly partitioned into 20 regions
 of interest (ROIs). This process increased the dataset size from 547 to
 10,940 images. Each ROI was resized to $224\times224$ pixels to match the
 input size of the ResNet50 model, and labeled images were converted into a
 single-channel 8-bit format. The resulting dataset was split into an 80/20
 ratio for training and validation, with 8,752 images used for training and
 2,188 for validation. <strong>Model
 Pipeline</strong> For our study, a joint model
 consisting of a DeepLabV3+ model with a ResNet50 backbone was used to
 enhance feature extraction for semantic segmentation tasks. DeepLabV3+ was
 chosen due to its strong performance in pixel-level segmentation tasks and
 its availability in MATLAB's Deep Learning Toolbox, which provided a
 straightforward implementation path. While newer architectures such as
 YOLO or SAM/SAM2 have demonstrated superior speed and performance,
 DeepLabV3+ remains highly effective in various segmentation applications,
 and it offers a reliable baseline for comparison in this study.
description: # Data from: High-throughput classification and quantification of skinning
 phenotype in sweet potatoes
 [https://doi.org/10.5061/dryad.v15dv4272](https://doi.org/10.5061/dryad.v15dv4272) ## Description of the data and file structure Data are provided for the machine learning model and pipeline, including the model's coefficients in Matlab, and regions of interest (ROIs) used for training and validation. ### Files and variables #### File: Skinning_ML_Pipeline.zip **Description:** The Skinning_ML_Pipeline.zip archive contains three directories and one MATLAB script: 1. Skinning_SP_Training.m: Script for loading training images, running model transfer learning, and converting the raw imagery data used for validation and testing. Certain cells can also be used for inference of new imagery, or inference can be run on the sample data provided. 2. Folder Train_and_validate_labels/ contains the labels for the training and validation images used for creating the model. The sub-folder "validate" contains the labels for the validation dataset (2188 images), and the sub-folder "train" contains the corresponding labels used for training (8752 images). 3. Folder Train and validate_images/ contains the RGB data for the training and validation images used for creating the model. The sub-folder "validate" contains the RGB images for the validation dataset (2188 images), and the sub-folder "train" contains the corresponding labels used for training (8752 images). 4. Folder Test_Images/ contains the data used for testing the model. Sub-folder "validate_test" contains the RGB images for testing, and "train_test" contains the labels for testing. 5. Folder Trained_model/ contains the ML model's coefficients for running inference on regions of interest. These coefficients are contained in "Trained_Model.mat." This is in Matlab file format, but it can be viewed or imported into Python using numpy. The file contains 6 cells. Each cell performs the following: 1. Cell 1: Transfer learning to a new model to determine new coefficients. If the published coefficients are used, one can start at cell 2 to load the existing coefficients for inference. 2. Cell 2: Evaluate the Model with Test Images based on the coefficients located in Trained_Model. 3. Cell 3: Implement the network and calculate skinning percentage from results. 4. Cell 4: Load masks to verify their uniformity and properties visually. 5. Cell 5: Calculate the skinning percentage to generate RMSE plots from the paper. 6. Cell 6: Generate confusion matrices and other metrics for the test dataset in units of pixels ## Code/software Shareable_Skinning_SP_Code_by_Zach_Bloom.m: Contains the code needed to train and validate the model, as well as running inference using the model's coefficients.