USDA Trial Document Questions

1. The Instrument Installation Process document is stricter than the approval process 

document? 

- Many of the requirements for an install seem to be significantly stricter than the 

actual approval process as per Instrument Approval Process? 

- Excel file comparing the two documents is attached

2. How is every trait calculated?

- Is there some documentation used by the Gold Standard graders which we could 

review to get more information about their process?

3. 95% within 20 marbling units seems like a very tight requirement, given the full range is 

1000 (is this correct?) units. How consistently does a USDA gold-standard human 

grader meet this requirement (say, vs. the average of two other independent human 

graders)?

- Is there scope to loosen parameters if it is shown that the three gold standard 

graders aren’t able to meet the parameters?

4. Regarding the performance requirements for ribeye area

- What formula is R2

- Is this intended to be invariant to linear transformation?

- The document references correlation and regression, could you elaborate?

5. What are the costs associated with an approval trial, how often is it reasonable to run 

one? This is to get an idea of whether it makes sense to use the trials themselves as the 

means of getting training data and testing/refining the camera system. Expert data is 

extremely valuable to us, so if we can reasonably pay for and use the trials as a basis for 

iterating on the product then that would allow us to develop more rapidly and with a 

greater degree of confidence. Is there a maximum number of trials allowed?

6. We are concerned about practical and IP implications of section 12.1 VERIFICATION 

OF APPROVED EQUATIONS that suggests that the instrument manufacturer must 

allow LP personnel access to the unencrypted software coding for an instrument 

immediately upon request. 

- Is it sufficient to provide the APK (install file) we are submitting for approval along 

with its SHA-1 hash?

- This would allow the regulator to use standard tools to confirm categorically that 

the installed version of the application is exactly identical to the provided/installed 

version.

- We can assist with finding an expert to validate the application
7. Instrument Approval Process specifies: “The technology provider will provide LP with a 

copy of the prediction equation used as well as the values for each variable in the 

prediction equation for each of the predicted marbling scores” (section 5.3.1.3)

- The primary components of our prediction systems are convolutional neural 

networks that operate on entire images. These algorithms use millions of 

parameters so a rolled out equation is nonsensical to a human

- Could you elaborate on what is the intended purpose of this rule so we can find a 

way to meet the intent of it?

8. Does our current data capturing method (video) comply with the Triple replacement and 

trigger method specified for data collection?

9. From section 5.3.1.4 – Quality control of images and data collected. “LP will review all 

data and images to determine if the data and images from the selected carcasses were 

accurately obtained. Technology providers must submit to LP the original image and the 

processed display screen image or thumbnail for each carcass in the test so that images 

can be reviewed to determine if proper image capture occurred.” 

- Does this mean we need to send the video for the test images and the 

segmentation mask for each carcass?

- Will USDA be removing the ones that might be of bad quality and not 

considering them?

10. Regarding 5.3.1.2 – Procedure for determining the Official Marbling Score, could we 

clarify how the process of splitting into calibration and test works? 

“Carcasses will be ranked by the final Official Marbling Score in ascending order by trial 

location and then every other carcass will be assigned to a calibration data set”.

11. What about updates to the handset’s software environment that aren’t at all involved with 

the data capture or prediction systems? Would that need to go through the update 

approval process? Scenarios including:

- Updating the Android operating system (important for security reasons; but 

involves proprietary Samsung/Google code we have no control over)

- Installing unrelated applications for user convenience or testing (less important 

but can be nice to have access to)

- Gmail, WhatsApp etc

- Updating the secondary application that handles barcode scanning, user login 

etc, perhaps to support a new type of barcode or enable better detection. Or the 

integration with a new processor.
12. There is no exclusion criterion for Ribeye Area, Rib fat, or Yield? In our experience we 

have seen significant variation in Gold Standard grades for these measures.

- Is it possible to include an exclusion criteria to ensure that the gold standard 

grades are accurate?

13. Can we collect gold standard data for the purpose of training models before the 60 day 

period is over?

14. Can we take images with a micro grid to prove ribeye and rib fat if there are 

discrepancies? Example image below.

15. Why does the Installation Document reference Instrument Grading Systems for Beef 

Carcasses but in Approval Process it’s stated that it replaces Instrument Grading 

Systems?