DOI: 10.5061/dryad.2280gb640
title: Reliability for: Dairy calf disbudding wounds

description: Disbudding, a painful procedure, permanently damages the horn-bud tissue
 either with a hot iron (heat) or caustic paste (chemical). The resulting
 wounds persist for an average of 7-9 wk for hot-iron or 14-18 wk for the
 caustic paste method. Throughout this healing process, disbudding wounds
 can be characterized by their tissue composition. To create consistency
 among our research team with identifying these stages of healing, we have
 developed supporting materials. This Dryad submission contains information
 for evaluating observer reliability when describing disbudding wounds in
 dairy calves. The wound scoring system and definitions we have used in
 previous research are included, along with a 110-photo test containing
 examples of the different tissue types. These photos are intended to be
 used to test observer reliability before data collection begins, during
 data collection, or afterwards. Included are: (1) the gold standard key
 for comparison, (2) a blank, formatted Excel sheet for data entry, and (3)
 an example response from an intern. Finally, the R code (kappa R package
 irr, version 0.84.1) used for calculating intra- and
 inter-observer reliability of disbudding wounds is presented. The outcomes
 of the analysis of the gold standard vs. gold standard and gold standard
 vs. an intern are shown. These data and resources can be used in future
 research to support reliable scoring among observers.
description: # Reliability for: Disbudding wounds after hot iron and caustic paste
 disbudding [Access this dataset on
 Dryad](https://doi.org/10.5061/dryad.2280gb640) These materials are
 intended to serve as a resource for researchers who want to score calf
 disbudding wound healing during future research projects. We have
 developed a photo test, gold standard scoring, and analysis for
 calculating inter observer reliability of disbudding wounds, which are all
 presented here. This code can be modified to calculate intra observer
 reliability as well. ## Description of the data and file structure *
 *DisbuddingWounds241014.pdf*; This file presents the wound tissues scored
 after hot-iron and caustic paste disbudding in dairy calves. A total of 9
 wound tissues are presented along with examples and their definitions. The
 reliability for identifying the presence and absence of each tissue type
 is evaluated in the RMD. * *DisbuddingWoundTest.zip*; 110 photos
 presenting examples of wounds after hot-iron and caustic paste disbudding,
 which should be scored to calculate inter observer reliability. The photos
 included in this test come from Holstein and Jersey calves that were
 disbudded using caustic paste at 3 d of age or a hot iron before 8 wk of
 age. Photos were collected throughout the entire healing process when
 wounds were scored as “fully healed” which takes an average of 10 wk for
 hot iron disbudding and 15.9 to 19.4 wk for caustic paste. At the time of
 disbudding, all calves received a local block and an NSAID. *
 *DisbuddingWoundScoringReliability_KEY_250227.xlsx*; This file presents
 the gold standard used to calculate inter observer reliability for each of
 the wound tissues scored. “Photo” corresponds to the numbering of the
 photos in the test, while 1 indicates a tissue type is present and 0
 indicates absence. The tissue definitions presented in Disbudding Wounds
 241014.pdf were used to score these photos. Within Excel, the top row has
 been frozen and filtered to order the rows by photo number to assist with
 reviewing data in this sheet. We find it is easiest to maintain a frozen
 top row to allow users to see the tissue types listed as they review the
 photo scoring. * *DisbuddingWoundScoringReliability_BLANK_250227.xlsx*;
 This file is a blank sheet which can be used to download and score the
 presence and absence (1/0) by observers for each of the 9 tissue types.
 Inter observer reliability can then be evaluated by comparing the
 observers scoring to the gold standard/key provided. The photo column
 corresponds to the photo number in the “Disbudding Wound Test” folder. For
 every photo and tissue type, a 0 or a 1 should be entered based on the
 absence or presence, respectively. If 0’s are not entered when tissues do
 not appear, the reliability will not be calculated properly. Within Excel,
 the top row has been frozen and filtered to order the rows by photo number
 to assist with filling data into this sheet. We find it is easiest to
 maintain a frozen top row to allow users to see the tissue types listed as
 they fill in 0/1. *
 *DisbuddingWoundScoringReliability_ScorerIntern1_250309.xlsx*; This file
 includes the presence and absence (1/0) scored by an intern labeled as
 Intern1 for anonymity, for each of the 9 tissue types where reliability is
 being evaluated. Within Excel, the top row has been frozen and filtered to
 order the rows by photo number to assist with filling data into this
 sheet. We find it is easiest to maintain a frozen top row to allow users
 to see the tissue types listed as they fill in 0/1. *
 *ReliabilityFor_DisbuddingWounds_Analysis.Rmd*; This RMarkdown file uses
 the Disbudding Wound Scoring Reliability KEY 250227.xlsx listed above to
 calculate the inter observer reliability for 9 wound tissues presented in
 Disbudding Wounds 241014.pdf. In this example, the Disbudding Wound
 Scoring Reliability ScorerIntern1 250309.xlsx is used to calculate
 reliability for Intern1. The file Disbudding Wound Scoring Reliability
 BLANK 250227.xlsx can be downloaded, filled in, and used to create
 reliability for other observers against our gold standard. Rmd files can
 be downloaded and run in R. This file is annotated. *
 *ReliabilityFor_DisbuddingWounds_Analysis.pdf*; This file is the PDF
 output of the corresponding RMarkdown file, and includes the code and
 output previews. This file is annotated. *
 *DisbuddingWoundReliabilityDiscrepancies_Intern1_250312.xlsx*; This file
 was created in the ReliabilityFor_DisbuddingWounds_Analysis.Rmd and
 exported to review disagreement in scoring between the gold standard and
 Intern1. The photo column shows the number of the photo scored where
 interns and the gold standard either “match” or have a “discrepancy”. We
 are most interested in reviewing the disagreement, so this Excel file
 specifically focuses on creating a visual representation of tissue types
 where there is disagreement between the observer and gold standard. If a
 photo had matches for each of the tissue types, it was excluded from this
 data frame before being exported. This file is exported from R with any of
 the discrepancies highlighted in bold red text to support ease of
 identification when reviewing in Excel. *
 *WoundReliabilityComparison_GoldandScorerIntern1_250312.xlsx*; This file
 was created in the ReliabilityFor_DisbuddingWounds_Analysis.Rmd and
 exported to check for disagreement in scoring between the gold standard
 and Intern 1. It shows the “Photo” number and the 0/1 score for both the
 Gold standard and Scorer Intern1 in separate columns for each tissue type
 presented in Disbudding Wounds 241014.pdf. ## Code/Software All analyses
 were run in R using RStudio. Software versions, including those for all
 packages used, are included in
 *ReliabilityFor_DisbuddingWounds_Analysis.Rmd*, and the associated PDF
 export.