# Short communication: Evaluation of a foot dirtiness scoring system for dairy cows

Juan Manuel Ariza, Pierre Levallois, Nathalie Bareille, Aurélie Arnoult, Raphaël Guatteo

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Juan Manuel Ariza, Pierre Levallois, Nathalie Bareille, Aurélie Arnoult, Raphaël Guatteo. Short communication: Evaluation of a foot dirtiness scoring system for dairy cows. Journal of Dairy Science, 2020, 103 (5), pp.4738-4742. 10.3168/jds.2019-17178. hal-02901374

HAL Id: hal-02901374

https://hal.inrae.fr/hal-02901374v1

Submitted on 13 Oct 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# Short communication: Evaluation of a foot dirtiness scoring system for dairy cows

J. M. Ariza,* P. Levallois, N. Bareille, A. Arnoult, and R. Guatteo

BIOEPAR, INRA, Oniris, La Chantrerie, F-44307, Nantes, France

## ABSTRACT

Lameness in dairy cattle is an important health and welfare concern directly affecting the economic benefits of farmers. Although foot dirtiness is a recognized risk factor for diseases inducing lameness, there is a lack of scoring systems that focus specifically on the foot. Therefore, the aim of this study was to evaluate the reliability of a 3-point dirtiness scoring system recently conceived for the assessment of the dirtiness present at the lower legs and claws of dairy cows. The scores of both the lateral and back views from 4 unfamiliar raters at 4 different times were used to evaluate interrater reliability (3 times) and intrarater scoring consistency (1 time). The findings of this investigation revealed that coefficients for interrater agreement (0.38, 0.62 and 0.63) and rating concordance (0.55, 0.75 and 0.74) of back-view scorings improved over time and were superior or similar to those obtained from lateral-view evaluations (coefficients for interrater agreement = 0.20, 0.35, and 0.69; coefficients for rating concordance = 0.48, 0.59, and 0.78). Additionally, the scoring consistency of the raters using the back-view grid was excellent (Cronbach's α = 0.93). The results suggest that the 3-point back-view dirtiness scoring grid may provide dairy consultants and farmers a reliable and easy tool for assessing cow foot dirtiness.

**Key words:** consistency, reliability, scoring, foot dirtiness

## Short Communication

Cow dirtiness is a recognized determinant for some of the most important multifactorial diseases affecting dairy cows, such as mastitis and lameness (Schreiner and Ruegg, 2003; Solano et al., 2015). Like mastitis, lameness may negatively affect the economic benefits of farmers and the welfare of cows (Bruijnis et al., 2012; Huxley, 2013), and therefore it represents a main

concern for the dairy cattle industry. Previous studies have shown that dirtiness of the udder, hocks, and feet can be associated with impaired udder health (Ellis et al., 2007; Sant'anna and Paranhos da Costa, 2011). Likewise, leg dirtiness was specifically related to an increasing prevalence of hock injuries (Zurbrigg et al., 2005) and the incidence of digital dermatitis (Relun et al., 2013). Additionally, environmental hygiene seems to play an important role in other infectious diseases such as interdigital dermatitis, heel horn erosion, and sole ulcers (Hultgren and Bergsten, 2001; Lombard et al., 2010; Knappe-Poindecker et al., 2013). Currently, foot dirtiness in cows is usually assessed through grids evaluating the entire leg (Hughes, 2001; Reneau et al., 2005; Cook, 2006) and not the foot specifically. These scoring systems assessing the entire leg could potentially score too low the true dirtiness affecting the foot. Consequently, our study aimed to evaluate the interrater reliability and intrarater consistency among new users of a recently developed lower leg and claw dirtiness scoring system (Guatteo et al., 2013) that is easy to implement on a regular basis.

In practice, evaluations using the foot dirtiness scoring system (FDS) should be performed while the cow is standing (i.e., at the locking yokes while feeding or in the milking parlor), and the dirtiest hind foot should be assessed from both the lateral and back views (Figure 1). To evaluate and compare the reliability and practicality of the scoring grid, 4 unfamiliar raters performed multiple different evaluations, including a field visit and different web forms. We scrutinized the data obtained from these scorings to evaluate (1) the correlation between the observations performed on both views of the foot (lateral and back), (2) the interrater reliability, (3) intrarater scoring consistency, and (4) whether the interrater reliability was enhanced over time when raters became more familiar with the grids.

For data analysis purposes, the internal consistency (correlation between lateral-view and back-view observations) of the scoring grid studied was assessed using the calculation of Cronbach's α statistic (Cα; Cronbach, 1951). The agreement between raters was assessed using Krippendorff's α calculation (Kα; Krippendorff,

<footnote>Received June 27, 2019.
Accepted November 5, 2019.
*Corresponding author: juan-manuel.ariza@oniris-nantes.fr</footnote>
<table><caption>Figure 1. Scoring grid to assess lower leg and claw dirtiness of dairy cows according to 2 views, lateral and back (Gyatteo et al., 2013).</caption><thead><tr><th rowspan="2">Dirtiness Score</th><th colspan="2">Lower Leg and Claw View</th><th rowspan="2">Foot Dirtiness Description</th></tr><tr><th>Lateral</th><th>Back</th></tr></thead><tbody><tr><td>1</td><td></td><td></td><td>Thin crust or absence of crust over the hoof wall and coronary band visible.</td></tr><tr><td>2</td><td></td><td></td><td>Thick crust expanding from the hoof wall and a small part of the coronary band. Dew claws covered by a crust but the area under the dew claws still visible.</td></tr><tr><td>3</td><td></td><td></td><td>Thick crust covering the hoof wall, the coronary band, the dew claws, and a large part of the metatarsus.</td></tr></tbody></table>

2004a). Additionally, due to the ordinal nature of the scoring system investigated, Kendall's coefficient (Wt) was calculated to evaluate the concordance between the

scores from multiple raters assessing the same foot. Finally, intrarater scoring consistency was assessed using $C\alpha$. For this investigation, $C\alpha \ge 0.70$, $K\alpha \ge 0.40$ to
0.60, and Wt ≥ 0.40 to 0.60 were considered acceptable (Bland and Altman, 1997; Krippendorff, 2004b). All data were analyzed using R (R Core Team, 2019).

Before starting the scoring, the 4 unfamiliar raters (veterinary students) were trained once using a photo gallery that included 72 foot photos encompassing different views and diverse levels of dirtiness. The first part of the investigation relied on a single herd visit where raters scored lactating Holstein cows with varied leg and claw colors using the FDS. One week later, raters were asked to perform the scoring on a web form that included photos (81 lateral views and 81 back views) of the foot of the cow's dirtiest leg during the previous field visit. The photos were taken approximately 50 cm from the dirtiest foot to obtain views similar to the photos on the grid (Supplemental Figure S1, https://doi.org/10.3168/jds.2019-17178).

Increasing the heterogeneity of our study sample could help us evaluate whether the scoring system could be implemented in herds of different profiles. Therefore, for the second part of the study, a new web form was created that included 180 foot photos (90 back views and 90 lateral views) from 90 Holstein cows of varied leg and claw colors; the photos were collected from 16 dairy herds to create a virtual herd. For the third part of the study, a subset of 26 photos (13 back views and 13 lateral views) from the virtual herd created for the second part of the study was used to investigate intrarater scoring consistency. The photos chosen for this new web form represented all score categories. First, raters scored the photos via the web form; then, 2 wk later, the order of the photos was modified and raters again filled out the web form. Finally, the effect of training on interrater agreement was investigated using the data recorded during the first 2 parts of the study. Thus, according to the time when raters performed scoring, 3 time points (field visit, web form created from the field visit, and virtual herd web form) were created and analyzed.

Our findings revealed that the correlation between lateral-view and back-view scorings was poor (Cα range = 0.19–0.53) independent of how the scoring was performed (field visit or web form; Table 1). By investigating the consistency between both view scores and the rater agreement of each view, we were able to provide evidence about whether the view affected the reliability of the FDS. When comparing the scorings performed during the field visits with those performed through the web forms, the interrater agreement was superior on web forms (Table 1). These results could be attributed to the fact that web form scorings are mainly affected by the quality of the photos, whereas during a field visit multiple factors (e.g., lighting, foot position at appraisal, unexpected movements of the animals,

environmental conditions of a farm) may influence the correct scoring of the foot. Moreover, the high quality of the web form photos could enhance the appraisal of the raters. Although for the scoring system studied the dirtiest foot is considered representative of the 4 feet of the cow, interrater agreement could be affected by the choices of the raters performing the scoring under field conditions. Otherwise, when comparing agreement between the lateral-view and back-view evaluations, interrater agreement was superior or equivalent for the back-view evaluations in both field visits and web forms (Table 1). At every scoring performed during the study, the proportions of score 3 (the dirtiest score on a scale of 1 to 3) recorded from lateral-view evaluations (field visit = 10%, field visit web form = 17%, virtual-herd web form = 17%) were lower than those recorded from back-view evaluations (field visit = 78%, field visit web form = 56%, virtual-herd web form = 45%; Figure 2). The differences in the distribution of scores according to the view evaluated may be due to the backside being prone to receiving more fecal discharge compared with the lateral side of the lower leg and claw. Thus, back-view evaluations might reflect the most relevant information about foot dirtiness. Additionally, this view can be easily and safely assessed during milking in herds using parallel and rotatory parlors or during routine feeding in herds using other milking systems. Furthermore, different foot disorders associated with dirtiness, such as digital dermatitis, are more frequently localized in the backside of the foot (Solano et al., 2016). Therefore, viewing from the back may be a more appropriate method for evaluating lower leg and claw dirtiness based on their relevance and the acceptable agreement evidenced by the FDS across this study.

As pointed out above, multiple management practices may affect the foot dirtiness of a given herd. For this reason, during the second part of the investigation, we evaluated whether the reliability of the scoring systems was affected by a heterogeneous dirtiness herd profile. We hypothesized that the reliability of the scoring grid studied did not vary between the scorings performed on the previously visited herd and the virtual herd created. According to our results, the interrater agreement of back-view observations remained unchanged and increased for the lateral view. These findings support an improvement in reliability that could be better explained by the training effect of raters than by the herd profile differences. However, the 2 herds (herd visited and virtual herd) included in this investigation may underrepresent the heterogeneity present in commercial dairy herds, and further studies are needed to highlight the role of foot dirtiness on lameness.

Regarding intrarater scoring consistency, raters were more consistent with their scoring when using the back-
<table><caption>**Table 1.** Agreement, rating concordance, and scoring consistency of 4 raters implementing a 3-point ordinal foot dirtiness scoring grid</caption><thead><tr><th>Assessing time</th><th>No.</th><th>α<sup>1</sup></th><th>95% CI</th><th>Wt<sup>2</sup></th></tr></thead><tbody><tr><td>Field visit</td><td rowspan="3">97</td><td></td><td></td><td></td></tr><tr><td>Lateral view</td><td>0.20</td><td>0.07 to 0.33</td><td>0.48</td></tr><tr><td>Back view</td><td>0.38</td><td>0.20 to 0.54</td><td>0.55</td></tr><tr><td>Lateral view vs. back view<sup>3</sup></td><td></td><td>0.33</td><td>0.20 to 0.45</td><td></td></tr><tr><td>Field visit web form</td><td rowspan="3">81</td><td></td><td></td><td></td></tr><tr><td>Lateral view</td><td>0.35</td><td>0.23 to 0.47</td><td>0.59</td></tr><tr><td>Back view</td><td>0.62</td><td>0.50 to 0.72</td><td>0.75</td></tr><tr><td>Lateral view vs. back view<sup>3</sup></td><td></td><td>0.22</td><td>0.07 to 0.38</td><td></td></tr><tr><td>Virtual-herd web form</td><td rowspan="3">90</td><td></td><td></td><td></td></tr><tr><td>Lateral view</td><td>0.69</td><td>0.62 to 0.77</td><td>0.78</td></tr><tr><td>Back view</td><td>0.63</td><td>0.52 to 0.72</td><td>0.74</td></tr><tr><td>Lateral view vs. back view<sup>3</sup></td><td></td><td>0.03</td><td>-0.17 to 0.23</td><td></td></tr><tr><td>Intrarater subset web form<sup>3</sup></td><td rowspan="3">26</td><td></td><td></td><td></td></tr><tr><td>Lateral view</td><td>0.64</td><td>0.46 to 0.82</td><td></td></tr><tr><td>Back view</td><td>0.93</td><td>0.9 to 0.97</td><td></td></tr></tbody></table>

view grid compared with the lateral-view grid (Table 1). Finally, across the investigation, interrater agreement improved along with rater experience. Acceptable interrater agreement measures were achieved after the second scoring time using the FDS back view and notably improved for the lateral views at the last scoring performed. Therefore, training seems to be an important step before the periodic implementation of the grid in

field conditions. Raters could be easily trained in a first session using a photo gallery and then across consecutive scorings, as implemented in this study. Whereas Kα considers only the agreement between raters, Wt considers the consequences of misclassifying the good hygiene (score = 0) of a foot as poor (score = 3) to be more serious than misclassifying it as fair (score = 2). Therefore, both coefficients are complementary and

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Distribution of the scores (1–3) among 4 raters according to the assessing time and the view assessed. See Figure 1 for definitions of dirtiness scores.</figcaption></figure>

<footnote n="1">¹Krippendorff's α statistic for measuring interrater agreement.</footnote>

<footnote n="2">²Kendall's coefficient statistic for measuring the concordance between the scorings from multiple raters.</footnote>

<footnote n="3">³Cronbach's α statistic for measuring the correlation between 2 tests measuring the same construct.</footnote>
allow a complete reliability assessment for determining
the usefulness of training (repetition) and the practical-
ity of the FDS.

Controlling foot dirtiness may be a useful comple-
mentary strategy in the prevention of lameness. This
investigation demonstrated the interrater reliability and
intrarater scoring consistency of the FDS (back view).
This scoring system may provide dairy consultants and
farmers a reliable tool for objectively assessing cow
foot dirtiness. Future studies assessing the effect of foot
dirtiness on lameness may benefit from this tool.

ACKNOWLEDGMENTS

The authors acknowledge the farmers and veterinary
students who contributed to this study. The authors
have not stated any conflicts of interest.

REFERENCES

Bland, J. M., and D. G. Altman. 1997. Statistics notes: Cronbach's alpha. BMJ 314:572. https://doi.org/10.1136/bmj.314.7080.572.

Bruijnis, M. R. N., B. Beerda, H. Hogeveen, and E. N. Stassen. 2012. Assessing the welfare impact of foot disorders in dairy cattle by a modeling approach. Animal 6:962-970. https://doi.org/10.1017/S1751731111002606.

Cook, N. 2006. The Hygiene Scoring Card. Accessed Jun. 27, 2019. https://www.vetmed.wisc.edu/dms/fapm/fapmtools/4hygiene/hygiene.pdf.

Cronbach, L. J. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16:297-334. https://doi.org/10.1007/BF02310555.

Ellis, K. A., G. T. Innocent, M. Mihm, P. Cripps, W. G. McLean, C. V. Howard, and D. Grove-White. 2007. Dairy cow cleanliness and milk quality on organic and conventional farms in the UK. J. Dairy Res. 74:302-310. https://doi.org/10.1017/S002202990700249X.

Guatteo, R., A. Arnoult, J.-L. Ménard, and N. Bareille. 2013. Elaboration of a scoring grid to assess feet cleanliness in dairy cattle and identification of risk factors in the winter period. Pages 379-382 in Rencontres Recherches Ruminants. Vol. 20. Rencontres autour des Recherches sur les Ruminants, Paris, France.

Hughes, J. 2001. A system for assessing cow cleanliness. In Pract. 23:517-524. https://doi.org/10.1136/inpract.23.9.517.

Hultgren, J., and C. Bergsten. 2001. Effects of a rubber-slatted flooring system on cleanliness and foot health in tied dairy cows. Prev. Vet. Med. 52:75-89. https://doi.org/10.1016/S0167-5877(01)00237-9.

Huxley, J. N. 2013. Impact of lameness and claw lesions in cows on health and production. Livest. Sci. 156:64-70. https://doi.org/10.1016/j.livsci.2013.06.012.

Knappe-Poindecker, M., M. Gilhuus, T. K. Jensen, K. Klitgaard, R. B. Larssen, and T. Fjeldaas. 2013. Interdigital dermatitis, heel horn erosion, and digital dermatitis in 14 Norwegian dairy herds. J. Dairy Sci. 96:7617-7629. https://doi.org/10.3168/jds.2013-6717.

Krippendorff, K. 2004a. Content Analysis: An Introduction to Its Methodology. SAGE Publications Inc., Thousand Oaks, CA.

Krippendorff, K. 2004b. Reliability in content analysis. Hum. Commun. Res. 30:411-433. https://doi.org/10.1111/j.1468-2958.2004.tb00738.x.

Lombard, J. E., C. B. Tucker, M. A. G. von Keyserlingk, C. A. Kopral, and D. M. Weary. 2010. Associations between cow hygiene, hock injuries, and free stall usage on US dairy farms. J. Dairy Sci. https://doi.org/10.3168/jds.2010-3225.

R Core Team. 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

Relun, A., A. Lehebel, M. Bruggink, N. Bareille, and R. Guatteo. 2013. Estimation of the relative impact of treatment and herd management practices on prevention of digital dermatitis in French dairy herds. Prev. Vet. Med. 110:558-562. https://doi.org/10.1016/j.prevetmed.2012.12.015.

Reneau, J. K., A. J. Seykora, B. J. Heins, M. I. Endres, R. J. Farnsworth, and R. F. Bey. 2005. Association between hygiene scores and somatic cell scores in dairy cattle. J. Am. Vet. Med. Assoc. 227:1297-1301. https://doi.org/10.2460/javma.2005.227.1297.

Sant'Anna, A. C., and M. J. R. Paranhos da Costa. 2011. The relationship between dairy cow hygiene and somatic cell count in milk. J. Dairy Sci. 94:3835-3844. https://doi.org/10.3168/jds.2010-3951.

Schreiner, D. A., and P. L. Ruegg. 2003. Relationship betweenuder and leg hygiene scores and subclinical mastitis. J. Dairy Sci. 86:3460-3465. https://doi.org/10.3168/jds.S0022-0302(03)73950-2.

Solano, L., H. W. Barkema, S. Mason, E. A. Pajor, S. J. LeBlanc, and K. Orsel. 2016. Prevalence and distribution of foot lesions in dairy cattle in Alberta, Canada. J. Dairy Sci. 99:6828-6841. https://doi.org/10.3168/jds.2016-10941.

Solano, L., H. W. Barkema, E. A. Pajor, S. Mason, S. J. LeBlanc, J. C. Zaffino Heyerhoff, C. G. R. Nash, D. B. Haley, E. Vasseur, D. Pellerin, J. Rushen, A. M. de Passillé, and K. Orsel. 2015. Prevalence of lameness and associated risk factors in Canadian Holstein-Friesian cows housed in freestall barns. J. Dairy Sci. 98:6978-6991. https://doi.org/10.3168/jds.2015-9652.

Zurbrigg, K., D. Kelton, N. Anderson, and S. Millman. 2005. Tie-stall design and its relationship to lameness, injury, and cleanliness on 317 Ontario dairy farms. J. Dairy Sci. 88:3201-3210. https://doi.org/10.3168/jds.S0022-0302(05)73003-4.

ORCIDS

J. M. Ariza e https://orcid.org/0000-0001-8525-7080

P. Levallois e https://orcid.org/0000-0002-3232-8209

N. Bareille e https://orcid.org/0000-0002-3105-7953

R. Guatteo e https://orcid.org/0000-0002-6658-522X