Back to Top [Skip to main content](#)
[Image](#)
Close

## Account

Logged in as:

username

- Dashboard

- Publications

- Account settings

- Log out

Access keys [NCBI Homepage](#) [MyNCBI Homepage](#) Main Content [Main Navigation](#)

Preview improvements coming to the PMC website in October 2024. [Learn More](#) or [Try it out now](#).

Search PMC Full-Text Archive Search in PMC

- Advanced Search

- User Guide

- Journal List

- Front Vet Sci

- PMC8328145

Other Formats

- PDF (2.9M)

Actions

- Cite

- Collections
  Add to Collections
○ Create a new collection

○ Add to an existing collection

Name your collection:

Name must be less than characters

Choose a collection:

Unable to load your collection due to an error

[Please try again](#)

Add Cancel

## Share

<figure><img src="image_1.png" /></figure>

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

Permalink

Copy

## RESOURCES

- Similar articles

- Cited by other articles

- Links to NCBI Databases

- [Journal List](#)

- [Front Vet Sci](#)

- PMC8328145

As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health.

Learn more: [PMC Disclaimer](#) | [PMC Copyright Notice](#)
*[image](#)

Front Vet Sci. 2021; 8: 689375.

Published online 2021 Jul 19. doi: [10.3389/fvets.2021.689375](https://doi.org/10.3389/fvets.2021.689375)

PMCID: PMC8328145

PMID: [34350229](https://www.ncbi.nlm.nih.gov/pmcsharesERM/)

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>
# Existence and Quality of Data on Control Programs for EU Non-regulated Cattle Diseases: Consequences for Estimation and Comparison of the Probability of Freedom From Infection

Egle Rapaliute,¹,* Annika van Roon,² Gerdien van Schaik,²,³ Inge

Santman-Berends,²,³ Xhelil Koleci,⁴ Madalina Mincu,⁵ Jörn

Gethmann,⁶ Beate Conrady,⁷,⁸,⁹ Tanja Knific,¹⁰ Jaka Jakob

Hodnik,¹⁰ John Berezowski,¹¹ Luís Pedro Carmo,¹¹ Aurélien

Madouasse,¹² Attila Tarpai,¹³ Anton Gerilovych,¹⁴ Alvydas

Malakauskas,¹ Blagica Sekovska,¹⁵ Christine Fourichon,¹²

Emmanouil Kalaitzakis,¹⁶ Franz-Ferdinand Roch,⁷ Hans Houe,⁹

Katarzyna Dudek,¹⁷ Kerli Mõtus,¹⁸ László Ózsvári,¹⁹ Lina Costa,²⁰

Maria Guelbenzu-Gonzalo,²¹ Madeleine K. Henry,²² Mentor

Alishani,²³ Nicola Pozzato,²⁴ Petter Hopp,¹³ Ramon Juste,²⁵ Sam

Strain,²⁶ Rene Mandelik,²⁷ Štefan Vilček,²⁷ Tiina Autio,²⁸ Lena-

Mari Tamminen,²⁹,† and Céline Faverjon³⁰,†

Author information Article notes Copyright and License information
PMC Disclaimer

## Associated Data

Data Availability Statement

## Abstract

Some European countries have successfully implemented country-specific control programs (CPs) for infectious cattle diseases that are not regulated or are regulated only to a limited extent at the
European Union (EU) level. Examples of such diseases include bovine viral diarrhea (BVD), infectious bovine rhinotracheitis (IBR), and Johne's disease (JD). The CPs vary between countries in the design and quality of collected data as well as methods used to detect infection and estimate prevalence or probability of freedom from infection. Differences in disease status between countries and non-standardized approaches to assess freedom from infection pose a risk for countries with CPs for non-regulated diseases as infected animals may influence the progress of the disease control or eradication program. The implementation of output-based standards allows estimation and comparison of the probability of freedom for non-regulated cattle diseases in European countries. The aim of the current study was to assess the existence and quality of data that could be used for estimating freedom from infection in European countries. The online data collection tool was sent to 32 countries participating in the SOUND control COST Action and was completed by 24 countries. Data on cattle demographics and data from CPs of IBR and BVD exist in more than 50% of the response countries. However, data describing risk factors and CP of JD was reported as existing in <25% of the countries. The overall quality of data in the sections on demographics and CPs of IBR and BVD were evaluated as “good”, but risk factors and JD data were mostly evaluated as “fair.” Data quality was considered less good mainly due to two quality criteria: accessibility and accuracy. The results of this study show that the quantity and quality of data about cattle populations and CPs are relatively similar in many surveyed countries. The outcome of this work provides an overview of the current situation in the European countries regarding data on EU non-regulated cattle diseases and will further assist in the development and implementation of output-based standards.

**Keywords:** animal health data, cattle, control programs, non-regulated diseases, output-based, proof of freedom

Introduction

Infectious animal diseases are known to be a risk to international trade and public and animal health. To benefit from international trade and provide legitimate protection from animal diseases and zoonoses, countries must comply with the guidelines of the World
Trade Organization Agreement on Sanitary and Phytosanitary Measures and the requirements of other standard-setting organizations, such as the World Organization for Animal Health (OIE) and/or the European Union (EU) (1-4). To demonstrate that a region or country is a safe trading partner for animals and animal products, it is necessary to prove freedom from disease (2). In the EU, international standards of surveillance to achieve desired proof of freedom have been developed for some important cattle diseases, e.g., bovine tuberculosis and bovine brucellosis (3). However, for other diseases listed as important for international trade by the OIE (5), such as Johne's disease (JD), infectious bovine rhinotracheitis (IBR), and bovine viral diarrhea (BVD), there are no or limited international standards for proving freedom of disease. Nevertheless, some European countries have successfully implemented country- or region-specific control programs (CPs) for these EU non-regulated diseases. Because of the lack of international standards, the CPs are very diverse, and their outputs are generally difficult to compare, impairing international trade (6-8).

In recent years, output-based standards have been successfully developed and implemented in animal health surveillance (9-15). They appear to be an attractive alternative to input-based standards for EU non-regulated infectious cattle diseases for several reasons. Input-based standards mean that, to be considered free from infection, countries must carry out specified surveillance activities, such as achieving a certain sampling frequency or a minimum sample size or using recommended diagnostic tests (16). On the other hand, output-based standards allow the flexibility to use a wide range of surveillance activities to reach a predefined output (i.e., probability of freedom from infection), supporting the development of cost-effective and efficient surveillance systems (15, 16). Countries with existing CPs for EU non-regulated diseases would then only need to make sure that their surveillance activities are able to achieve a certain level of confidence of freedom (i.e., output) without changing their whole surveillance system to meet prescribed surveillance strategies (i.e., input) (16). The output-based standards more easily adapt to country-specific conditions and better reflect the country-specific disease status (13).
However, the development and implementation of output-based
standards to assess the probability of freedom from infection come
with challenges. Methods developed for demonstrating freedom
from infection using multiple complex data and surveillance
activities include scenario tree models and Bayesian models (12, 17,
18). These methods require a large amount of good-quality data to
accurately model confidence of freedom from infection (19). The
required data have been described before in projects, such as STOC
free (Surveillance Tool for Outcome-based Comparison of FREEdom
from infection) and RISKSUR (Risk-based animal health surveillance
systems), which worked on developing and encouraging output-
based standards for animal health (7, 20). Such data include a
broad spectrum of information describing the cattle industry,
disease introduction risks, biosecurity levels, and existing disease
control programs. Only a small portion of these data are routinely
collected by the European Commission [e.g., Animal Disease
Notification System, Trade Control and Expert System (TRACES),
and the OIE World Animal Health Information System] for
epidemiological analysis of disease outbreaks, risk analysis, or
general statistical information. At the time of writing this paper, the
availability of the remaining portion of the data needed to estimate
the probability of freedom from infection is unknown. This is
especially a concern as good-quality data are likely to be more
available for EU regulated diseases compared with EU non-
regulated diseases. In addition, even though EU member countries
are obliged to collect some of these data on a regular basis, methods
of collection and sources of data most likely differ. Recent results
from the SIGMA project provide a good overview of the diversity of
the animal health data sources network in the EU (21), but the level
of heterogeneity in the data collected for different CPs in different
countries remains unknown. Understanding the data heterogeneity
is the first step toward the use of output-based standards for
proving freedom from EU non-regulated infectious diseases.

The COST Action (CA17110) “Standardizing Output-based
surveillance to control Non-regulated Diseases in the EU” (SOUND
control) aims to support output-based disease surveillance
initiatives and develop a framework that could be used to estimate
the confidence in freedom from EU non-regulated infectious cattle
diseases (22, 23). SOUND control covers 32 countries and provides
a great opportunity to assess at a large scale the data currently available for estimating the probability of freedom from infection for EU non-regulated cattle diseases and provide recommendations to support the future development of output-based standards.

Our study was conducted within the scope of the SOUND control Working Group 2 (WG2) activities (22) and aimed to (i) provide an overview of the existence of potential data required as inputs for estimating freedom from infection in the 32 Action member countries, (ii) evaluate and compare the quality of these data using a standardized approach, and (iii) review data sources of available data. JD, IBR, and BVD are among the diseases most frequently targeted by CPs implemented in the Action member countries (23). Therefore, we used these three EU non-regulated diseases as case studies.

# Materials and Methods

## Online Data Collection Tool

A thorough description of the online data collection tool, including its development, key lessons learned during the process, and definitions of the variables, can be found in van Roon et al. (24). The online data collection tool was designed using LimeSurvey software (25).

Data from the previous year (or the most recent available) were requested for data collection. Briefly, the online data collection tool was divided into two main parts with four sections and (Figure 1):

[image]

Open in a separate window

Figure 1

The structure of the online data collection tool that was used to overview data related to control programs of infectious cattle diseases among 32 SOUND control COST Action member countries. *, mandatory part; n, number of variables *per section*.

• I. *General information*. This section included five basic
questions about the time period of the data assessed in the questionnaire, country, contact information of the respondent, and definitions of dairy and beef cattle.

## Part 1:

* **II. Demographics.** This section included 10 questions about the cattle population in the country or region considered in the questionnaire: number of cattle and herds, average herd size, number of births, number of herds with calves, cattle density, and number or percentage of farms with small ruminants and mixed farms (mixed farms are defined as all dairy herds that also have a type of beef cattle, such as veal calves, suckler cattle, etc.).

* **III. Risk factors.** This section included 18 questions about possible risk factors for disease introduction into a cattle herd, such as purchasing, grazing, breeding, housing of calves, control and management of manure, rodent and vector control, transport, disinfection, and equipment on the farm.

## Part 2:

* **IV. Disease control programs and testing strategies for JD, IBR, and BVD.** For each disease, respondents were asked to indicate if a CP for the disease existed in their country. A positive response was followed by seven questions on the number of herds participating in the CP, number of herds tested for the selected disease, animal-, and herd-level prevalence of the selected disease, herds that have a free status for the selected disease, and the number of herds that identified infected animals.

Each question was followed by four additional subquestions related to data existence, data quality, quantitative data, and data sources used to obtain information (Figure 1). The only mandatory question was about the existence of data. Data quality was evaluated using the "Data quality evaluation tool" presented in van Roon et al. (24), which included four criteria (Figure 1B):

* **Accessibility.** The availability of data. This criterion is important because it provides information about how data
were collected, for what purpose, and how readily the data
can be accessed (e.g., data exist but can be accessed only by
combining multiple data sources).

*   **Timeliness.** Often described as “up-to-dateness,” it varies depending on the purpose for which the information is required. It is important to evaluate the timeliness to determine whether the information reflects the most recent information.

*   **Completeness.** Refers to whether there are missing and/or unknown data fields in the database (e.g., for the variable "number of cattle/herds in territory," completeness would represent the percentage of farmers entering this information in the data base: 85% of all farmers having filled in the data means completeness of this variable is 85%).

*   **Accuracy.** Aims at assessing to what extent the stored values for an object are the correct values (e.g., when data validation procedures are implemented on a regular basis, it is more likely that data is accurate).

Data quality was evaluated using a standardized scoring method (Figure 1B) (24). First, for each quality criteria, a score of three (“good”), two (“fair”), or one (“poor”) was given. Second, the overall quality of the available data was then calculated using the sum of the scores obtained for each quality criterion. Overall quality was also defined as “good” (sum 9–12), “fair” (sum 5–8), or “poor” (sum 1–4).

In addition to data quality, the data source for each question was collected. Respondents were able to choose one or multiple answers from a predefined list of potential data sources, including the option “other,” for which the respondent provided the name of the data source (Figure 1D). Participants were also allowed to submit quantitative data associated with each question (not analyzed in this study) (Figure 1C). Data relating to all cattle (i.e., dairy and beef cattle together) were requested, but it was also possible to submit data separately if needed.

**Data Collection and Management**

The SOUND control consortium included representatives by
country: one management committee member and one or more
management committee substitutes and/or workgroup members.
Members who were participating in the workgroups related to data
about CPs, one person per country, were responsible for providing
the data for their respective country. Almost all participants had a
doctoral degree in the field of veterinary sciences or epidemiology
and most of them worked in the field of cattle health with a focus
on infectious diseases or in surveillance and control of cattle
infectious diseases. Depending on their knowledge, the participants
could either collect all information to fill in the questionnaire
themselves or ask others in their country to help them fill in the
questionnaire. Thirty-two SOUND control member representatives
were invited by email in July 2020 to fill in the online data
collection tool. Three reminders were sent at the end of August,
September, and November 2020. The deadline to submit data in the
data collection tool was closed in December 2020.

Fully or partially completed questionnaires were extracted from LimeSurvey software to Microsoft Excel 365 and then imported into the R Statistical software, version 3.6.2 (26, 27) for analysis. Descriptive analysis was performed using the “dplyr” package in R (28). Data analysis was structured by country and type of question to calculate and assess (i) the number and proportion of existing data, (ii) the overall quality and quality by criterion of existing data, and (iii) the variety of sources used to obtain data. The R package “ggplot2” was used to visualize the results (29).

## Results

### Response Rate

Twenty-four out of 32 of the SOUND control countries completed the online data collection tool (Figure 2). Twenty-two countries fully completed the tool (“Full response”), one country completed only the first part (“Response: PART 1”), and another country completed only the second part (“Response: PART 2”). One country did not fully complete any part of the tool (“Incomplete response”), and seven countries did not respond to the invitation (“No response”). Twenty-two countries reported providing the latest data
of 2019–2020, and two countries provided older data (from 2017 to
2018). Regarding existence of CPs, 15 countries reported the
existence of a CP and answered questions about IBR, 14 for BVD,
and 11 for JD.

[image]

Open in a separate window

Figure 2

The response to an online questionnaire study on data collection related to EU non-regulated cattle diseases among 32 SOUND control COST Action member countries.

Definitions of Dairy and Beef Cattle

All descriptions of dairy cattle involved characteristics related to
milk production and breed, e.g., “deliver milk,” “used in dairy
production,” “pure dairy cattle breed,” “farms main income from
milk production.” The definitions of beef cattle were less specific.
Three countries indicated that there is no definition of beef cattle in
their country. When a definition was available, beef cattle were
often reported as “veal, beef, and fattening cows,” but sometimes as
“all cattle excluding dairy.” Seven countries reported official
definitions of dairy and beef. However, the rest of respondents did
not specify that the provided definitions were official.

The Existence of Data

More than 70% of the cattle demographics data investigated in our
study were reported as existing in the 24 response countries (Table
1). However, this was only true for 24% of data regarding risk
factors (Table 1). Instead, data were reported as “not existing”
(38%) or “unknown” (38%).

Table 1

Results of the response to questions on existence of data relevant for
estimating freedom from infection in cattle in an online
questionnaire study among 24 European countries.
<table><thead><tr><th>Section</th><th></th><th>Data exist<sup>b</sup></th><th>No existing data<sup>b</sup></th><th>Unknown<sup>b</sup></th></tr></thead><tbody><tr><td>II. Demographics</td><td>(n = 24)</td><td>75.0% (18)</td><td>8.3% (2)</td><td>12.5% (3)</td></tr><tr><td>III. Risk factors</td><td>(n = 24)</td><td>25.0% (6)</td><td>37.5% (9)</td><td>37.5% (9)</td></tr><tr><td rowspan="3">IV. Disease control programs<sup>a</sup></td><td>JD (n = 11)</td><td>36.4% (4)</td><td>27.2% (3)</td><td>36.4% (4)</td></tr><tr><td>IBR (n = 15)</td><td>66.7% (10)</td><td>13.3% (2)</td><td>20.0% (3)</td></tr><tr><td>BVD (n = 14)</td><td>71.5% (10)</td><td>21.4% (3)</td><td>7.1% (1)</td></tr></tbody></table>

Open in a separate window

The answers are presented aggregated per section in the questionnaire.

<sup>a</sup>Number of the response countries that have the respective control program in place.

<sup>b</sup>Percentage of countries that have chosen the answers “Data exist,” “No existing data,” “Unknown” out of the number of respondents in each section of the online data collection tool [number of countries/number of responding countries (n)].

Data relating to IBR and BVD CPs were reported as existing in more than 65% of the countries with CPs for these diseases (12 and 11 countries, respectively), and existence of data related to JD CPs was much lower (36%) (Table 1).

The types of data existing in each responding country are presented in Figure 3. All responding countries reported having data about the number of cattle although only 30% of the countries had information about mixed farms. In the section of risk factors, around 75% of the countries reported purchase data as existing, and the rest of the data were “unknown” and “not existing.” Almost all the data about CPs of BVD and IBR were reported as existing in around 75% of the countries with the implemented CPs for these diseases. For JD, on the other hand, data about the CP existed in 50% of those countries with a CP
[image]

Open in a separate window

Figure 3

The existence of data relevant for estimating probability of freedom from infection in cattle in an online questionnaire study among 24 European countries. The answers are presented by country and variable. \*free from disease status. FI, Finland; SK, Slovakia; CH, Switzerland; HU, Hungary; IE, Ireland; NIR, Northern Ireland (UK); SI, Slovenia; NO, Norway; SCT, Scotland (UK); SE, Sweden; ES, Spain; IT, Italy; NL, Netherlands; EE, Estonia; DE, Germany; LT, Lithuania; AT, Austria; RO, Romania; AL, Albania; UA, Ukraine; GR, Greece; PL, Poland; MK, North Macedonia; KS, Kosovo.

## The Quality of Existing Data

The results about the overall quality of existing data in SOUND control countries are presented in Table 2. More than 60% of the existing demographics and disease control program data was evaluated by the respondents as “good,” and a small proportion of these data were evaluated as “fair” or “poor.” The overall data quality was lower for data related to risk factors: only 33% were assessed as “good,” and 50 and 17% of risk factors data evaluated as “fair” and “poor,” respectively.

## Table 2

Results of the response to questions on overall quality of data relevant for estimating freedom from infection in cattle in an online questionnaire study among 24 European countries.

<table><thead><tr><th rowspan="2">Section</th><th rowspan="2"></th><th colspan="3">Overall quality</th></tr><tr><th>Good<sup>b</sup></th><th>Fair<sup>b</sup></th><th>Poor<sup>b</sup></th></tr></thead><tbody><tr><td>II. Demographics</td><td>(n = 18)</td><td>77.8%<br>(14)</td><td>16.7%<br>(3)</td><td>5.5% (1)</td></tr><tr><td>III. Risk factors</td><td>(n = 6)</td><td>33.3%<br>(2)</td><td>50.0%<br>(3)</td><td>16.7%<br>(1)</td></tr></tbody></table>
<table><tr><td rowspan="3">IV. Disease control programs<sup>a</sup></td><td>JD</td><td>(n = 5)</td><td>60.0% (3)</td><td>20.0% (1)</td><td>20.0% (1)</td></tr><tr><td>IBR</td><td>(n = 12)</td><td>58.3% (7)</td><td>25.0% (3)</td><td>16.7% (2)</td></tr><tr><td>BVD</td><td>(n = 11)</td><td>72.7% (8)</td><td>18.2% (2)</td><td>9.1% (1)</td></tr></table>

Open in a separate window

The answers are presented aggregated per section in the questionnaire.

<sup>a</sup>Number of the response countries that have the respective control program in place.

<sup>b</sup>Percentage of countries that had existing data and chose the answer “Good,” “Fair,” “Poor” for each variable in the online data collection tool out of the number of respondents in each section of the online data collection tool [number of countries/number of responding countries (n)].

The results were quite consistent within each country in terms of data quality in all four sections of the data collection tool (Figure 4). Around three quarters of demographics data in all response countries were evaluated as “good.” However, the quality of the average number of births and cattle per herd rated lower in more than half of the countries. A reverse picture of the quality score can be seen in the risk factors section as only data describing cattle purchase was consistently evaluated as “good” in all countries but one. Data on other existing risk factors was given lower quality scores.

[image]

Open in a separate window

Figure 4

The overall quality of data relevant for estimating probability of freedom from infection in cattle in an online questionnaire study among 24 European countries. The answers are presented by country and variable. *free from disease status. FI, Finland; SK, Slovakia; CH, Switzerland; HU, Hungary; IE, Ireland; NIR, Northern Ireland (UK); SI, Slovenia; NO, Norway; SCT, Scotland (UK); SE, Sweden; ES, Spain; IT, Italy; NL, Netherlands; EE, Estonia; DE,
Germany; LT, Lithuania; AT, Austria; RO, Romania; AL, Albania; UA, Ukraine; GR, Greece; PL, Poland; MK, North Macedonia; KS, Kosovo.

Data quality for all evaluated criteria are presented in **Table 3**. An average percentage of “good” quality data in each section is consistent by criterion. The lowest quality score for data from all sections was the score for completeness. On the other hand, timeliness data were given the highest quality score. Additionally, among the disease CPs, the quality of BVD data was evaluated highest and JD lowest within all four quality criteria.

**Table 3**

Results of the response to questions on quality of data by criterion relevant for estimating freedom from infection in cattle in an online questionnaire study among 24 European countries.

<table><thead><tr><th rowspan="2">[image]</th><th rowspan="2"></th><th colspan="3">Accessibility</th><th colspan="3">Timeliness</th></tr><tr><th>Good<sup>b</sup></th><th>Fair<sup>b</sup></th><th>Poor<sup>b</sup></th><th>Good<sup>b</sup></th><th>Fair<sup>b</sup></th><th>Poor<sup>b</sup></th></tr></thead><tbody><tr><td rowspan="2">II. Demographics</td><td>(n = 18)</td><td>55.6% (10)</td><td>33.3% (6)</td><td>11.1% (2)</td><td>55.6% (10)</td><td>38.9% (7)</td><td>5.5% (1)</td></tr><tr><td>(n = 6)</td><td>16.7% (1)</td><td>50.0% (3)</td><td>33.3% (2)</td><td>50.0% (3)</td><td>16.7% (1)</td><td>33.3% (2)</td></tr><tr><td rowspan="3">IV. Disease control programs<sup>a</sup></td><td>JD (n = 4)</td><td>50.0% (2)</td><td>25.0% (1)</td><td>25.0% (1)</td><td>50.0% (2)</td><td>25.0% (1)</td><td>25.0% (1)</td></tr><tr><td>IBR<sup>c</sup> (n = 10)</td><td>77.8% (7)</td><td>11.1% (1)</td><td>11.1% (1)</td><td>80.0% (8)</td><td>10.0% (1)</td><td>10.0% (1)</td></tr><tr><td>BVD (n = 10)</td><td>80.0% (8)</td><td>10.0% (1)</td><td>10.0% (1)</td><td>80.0% (8)</td><td>10.0% (1)</td><td>10.0% (1)</td></tr></tbody><tfoot><tr><td colspan="2">[image]</td><td colspan="3">Accessibility</td><td colspan="3">Timeliness</td></tr><tr><td colspan="2"></td><td colspan="3">Good<sup>b</sup> Fair<sup>b</sup> Poor<sup>b</sup></td><td colspan="3">Good<sup>b</sup> Fair<sup>b</sup> Poor<sup>b</sup></td></tr></tfoot></table>
<table><tbody><tr><td>II. Demographics</td><td>(n 61.1% 27.8% 11.1% 50.0% 44.4% 5.6%<br>= (11) (5) (2) (9) (8) (1)<br>18)</td></tr><tr><td>III. Risk factors</td><td>(n 16.7% 50.0% 33.3% 33.3% 50.0% 16.7%<br>= (1) (3) (2) (2) (3) (1)<br>6)</td></tr><tr><td>IV. Disease control programs<sup>a</sup></td><td>JD (n 50.0% 25.0% 25.0% 50.0% 25.0% 25.0%<br>= (2) (1) (1) (2) (1) (1)<br>4)</td></tr><tr><td></td><td>IBR (n 70.0% 20.0% 10.0% 50.0% 50.0% 0.0%<br>= (7) (2) (1) (5) (5) (0)<br>10)</td></tr><tr><td></td><td>BVD (n 70.0% 20.0% 10.0% 60.0% 30.0% 10.0%<br>= (7) (2) (1) (6) (3) (1)<br>10)</td></tr></tbody></table>

Open in a separate window

The answers are presented aggregated per section in the questionnaire.

<sup>a</sup>Number of the response countries that have the respective control program in place.

<sup>b</sup>Percentage of countries that had existing data and chose the answer “Good,” “Fair,” “Poor” by criteria and each variable in the online data collection tool out of the number of respondents in each section of the online data collection tool [number of countries/number of responding countries (n)].

<sup>c</sup>For accessibility n = 9.

## Variety of Sources Used to Obtain the Same Data

The respondents used a variety of sources to assess and obtain the requested data. On average, two different sources of data were selected to answer one question. The most selected data source option in all sections was “other” followed by “national animal health databases” (Figure 5). The sources of data most frequently reported under the category “other” were agriculture statistics, central statistics databases, livestock registration databases, national animal traceability databases, and CPs databases.
[image]

Open in a separate window

Figure 5

Sources used to obtain data relevant for estimating probability of freedom from infection in cattle in an online questionnaire study among 24 European countries. The answers are presented aggregated per part in the questionnaire.

Discussion

Our study provides the first overview of data availability and quality related to estimating freedom from infection in 24 European countries. In addition, an overview of data availability and quality regarding control programs implemented for three major EU non-regulated cattle diseases, i.e., BVD, JD, and IBR was provided. Previous similar studies that aimed to describe the cattle sector or disease CPs in Europe have included fewer countries (6, 30). As participation in this study was voluntary, not all invited countries fully completed the questionnaire; however, the response rate was high (75%, 24 out of 32 invited countries). Nevertheless, the response rate was likely influenced by the COVID-19 pandemic and could even have been higher in another situation because many veterinary epidemiologists or veterinary public health specialists involved in the project were also actively participating in the emergency response to the pandemic.

The Availability of Data for Probability of Freedom From Disease Estimates

A high proportion of the requested data about cattle demographics (75%) was reported as existing in the response countries. This section included general statistical information about the cattle sector, i.e., average herd size, number of calves born, etc. As having a computerized database for tracing individual bovines is a requirement within the EU (EC 1760/200), it is expected that this data is routinely collected within the response countries (31). Based on the responses, it appears that sufficient information about general demographics is available to describe the cattle industry
within the responding countries and to use these data as parameters for modeling freedom from disease (e.g., average herd size, number of animals). In contrast, only around one fourth (24%) of the requested data about risk factors were reported to exist. The existing data on risk factors were mainly related to cattle purchase, which was homogenously reported as existing in response countries. This is most likely because these data are mandatory to report under guidelines of the European Commission TRACES system (32). Data about less regulated risk factors for disease introduction, such as herds involved in communal grazing, shared transport, etc., were, on the other hand, mostly non-existent or unknown. Risk factors for introduction of disease play an important role in estimating the probability of freedom (16). To facilitate future development of output-based standards for CPs, more data related to this topic need to be collected on a large scale in a systematic manner and made available for scientific use. Nevertheless, animal movement data (i.e., number of animals imported), and prevalence of infection in the farm of origin might be sufficient to obtain a first estimate of freedom from infection as seen, for example, in a study defining output-based standards for tuberculosis in farmed deer (11). Given the amount of data available, future models developed to compare outputs of various CPs could focus on the usage of nearly any data describing cattle demographics and cattle purchase data for risk of introduction of disease (16). However, although the introduction of cattle into a herd has been reported as an important risk factor for JD, IBR, and BVD in several studies (33–35), other risk factors, such as participation in shows, grazing, and calving pen systems, are also described as important for estimating freedom from infection (33). Our results show that very few countries in Europe have access to these data on a national or regional level.

Many countries have implemented CPs for the three diseases considered in our analysis. Most data in those countries were reported as existing for BVD (72%) and IBR (66%) and less for JD (33.8%). Six of the countries with CPs claimed having freedom from IBR, four from BVD, and two for JD (23). Countries that reported having CPs in place and/or being free from the disease also reported noticeably more data existing than countries with no CPs and/or no free status in sections on demographics and risk factors. Knowing that demonstrating freedom from disease requires
sufficient scientific evidence that the disease is truly absent in the country, we could assume that those countries have developed efficient systems collecting more disease-specific data that is required to monitor and establish those statuses (36, 37).

## The Quality of Data Available for Probability of Freedom From Disease Estimates

Our results show that 50% of existing data were evaluated as “good” for all four quality criteria except for data related to risk factors (Table 3). In general, if the input data for freedom from disease estimations are good quality, results of those estimations could also be considered as good and accurate.

The criterion “timeliness” was evaluated as “poor” or “fair” by almost 40% of the responding countries meaning that a lot of the available data were not updated on a regular basis and do not reflect the current situation of the country, which may be an issue when estimating the probability of freedom from infection. Similarly, about 45% of the data were assessed as “fair” or “poor” in terms of accessibility, meaning that the data were not freely available for use. In addition, almost 50% of the data were evaluated as having poor or fair accuracy. Such lower evaluation indicates that data may be collected without any or only some validation procedures applied, leading to less credible information. Finally, more than half of the data had poor or fair completeness. This result could indicate that the data we aimed to assess likely come from sources where such information is not mandatory to report, leading to incomplete data sets and some missing values.

Ideally, all four data quality criteria evaluated in our study should be good quality to get as accurate as possible freedom from infection estimates. However, as mentioned in our results, completeness and accuracy were generally evaluated lower compared with timeliness and accessibility. Low data quality in terms of completeness and accuracy is significant as using data that comes from incomplete data sets and/or data that does not reflect the true values could lead to inaccurate estimates. Poor accessibility and timeliness, on the other hand, do not impact the ability to accurately estimate freedom of disease. However, they pose
practical difficulties as data is not easy to obtain and does not reflect the most current or “real-time” probability of freedom from disease.

The overview about the existence and quality could be used as an indicator of which data are easily accessible and which are not. In addition, existing data evaluated as “poor” or “fair” should be interpreted with care when modeling freedom from infection, and care should be taken to explore the uncertainty associated with the outputs of the model. If the quality of data is considered especially important in output-based surveillance, and high-quality data are unavailable, actions should be taken to address this gap. However, assessing the requirements was not part of this study. Still, this study does provide an indication of the gaps and possibilities for improvement when aiming to collect comparable data from many European countries.

## Future Perspectives for the Online Data Collection Tool and Collected Data

Our online data collection tool was initially converted from Excel spreadsheets to an online questionnaire using survey-making software “Limesurvey.” However, it is difficult to assess how sustainable this form of data collection is. One option to consider would be to move toward an individual database-like tool, to which countries would be able to submit data and receive the outcome. Another prospect could be to optimize the data-collection process and collect required data from primary sources directly to the database or model, similar to the results of the SIGMA project (21). In addition, data that were reported as not existing in most of the countries, i.e., more than 50%, could be excluded from the data collection tool as such data would not support a wide implementation of output-based standards in Europe. To fill this gap, further work of SOUND control will focus on identifying the data that is necessary for output-based surveillance, and a joint research agenda for future research will be developed. Future work should also consider availability and validity of diagnostic tools as well as inherent differences in transmission dynamics among the diseases in question. The questions on diagnostic strategies in this study could potentially provide a basis for this, but a higher
response rate would be required to evaluate them. Finally, in this
data collection, respondents were able to voluntarily submit
quantitative data that was not analyzed in this study (Figure 1C).
Quantitative data that were submitted by respondents will further
assist in developing and testing the first models for probability of
freedom from disease in an output-based framework.

Limitations of the Study

The current study has limitations in the assessment of the existence
and quality of data that could lead to possible bias. First, when a
respondent reports "no existing data," the information may exist
although it was unknown to the respondent. Similarly, "unknown"
data may actually reflect non-existing data. However, if data exist
and were easily accessible, the respondent would have most likely
found it. As such, both "unknown" and "non-existing" data may
represent not easily accessible or not well-described data.
Nevertheless, this data collection was conducted within the
framework of SOUND control, and our targeted respondent in each
country are experts in the field of animal health surveillance, which
increases the likelihood of the respondent being aware of existing
data sources. Second, uncertainties in data quality evaluation could
lead to false final outputs of the models (i.e., probability of freedom
from infection), e.g., when data was mistakenly evaluated as good.
This could also be an issue when discussing whether the probability
of freedom from infection reflects the true situation of the country.
Although respondents provide definitions of each quality criteria
when assessing them, it is possible that they were interpreted
differently by different respondents. Observed difficulties in data
quality evaluation were related to the requested data being
available from several sources, meaning that one source with the
data of interest was easily accessible, however containing less
accurate data compared with another, less accessible source with
more accurate data. In cases such as this, it was unclear which data
source to select and provide scores in the data quality evaluation
tool. Thus, data evaluations (as good/fair/poor) should not be used
as absolute values and be interpreted with care, and future studies
further validating consistency of answers to the criteria would be
useful. Finally, the results of the current study mainly represent
country-level data on the cattle industry, risk factors for disease
introduction, and CPs relevant to non-regulated cattle diseases.
Therefore, respondents indicated that some data is collected on a
regional level, but there is no centralized database from which to
take the relevant data. In future studies, this data collection
framework could be used to collect the data related to non-
regulated cattle diseases in order to have a summarized overview of
data originating from multiple sources.

Conclusion

With data from 24 EU countries, our work provides an overview of
the current situation in Europe in terms of data related to the EU
non-regulated cattle diseases. This study further identified gaps in
data availability, such as risk factor occurrence, which indicates
where further work is needed. A standardized system of output-
based surveillance could offer valuable evidence of animal health
status to countries engaging in the trade of live cattle. For this
approach to be optimized, it is necessary that those countries that
would benefit from the information that such a system can provide
should take steps to collate and share relevant data so that the
estimates are as accurate as possible and the system achieves its
potential. Overall, this work provides input for the next a step
toward an output-based framework.

Data Availability Statement

The raw data supporting the conclusions of this article will be made
available by the authors, without undue reservation.

Ethics Statement

This study was conducted within the COST Action (CA 17110),
Standardizing Output-based surveillance to control Non-regulated
Diseases in the EU (SOUND control) that was approved by the EU
Committee of Senior Officials through written procedure on 13
April 2018 [Memorandum of Understanding (MoU) (22)]. The
members of SOUND control that were involved in the current study
formally accept the Action's MoU and did not require the study to
be reviewed or approved by an ethics committee. In addition, the
members of SOUND control who provided the data used for the
study are co-authors of this paper. They have all reviewed and
approved the content of the manuscript.

Author Contributions

ER prepared the manuscript, which was revised primarily by CFa, L-MT, GS, and AR. Conceptual contributions were made by IS-B, XK, and MM. Provision of data for the study and editing the final version of the manuscript were made by JG, BC, TK, JH, JB, LPC, AMad, AT, AG, AMal, BS, CFo, EK, F-FR, HH, KD, KM, LÓ, LC, MG-G, MH, MA, NP, PH, RJ, SS, RM, SV, and TA. All authors contributed to the article and approved the submitted version.

Conflict of Interest

CFa was employed by the company Ausvet Europe. The remaining
authors declare that the research was conducted in the absence of
any commercial or financial relationships that could be construed as
a potential conflict of interest.

Acknowledgments

We thank Dinu Gavojdian (Research and Development Institute for Bovine Balotesti, Romania), Erja Tuunainen (Animal Health ETT, Finland), Ian Hutchinson (Epidemiology Research Unit, Scotland's Rural College, United Kingdom), Jožica Ježek, Matjaž Ocepek, Jože Starič, Ivan Toplak (Veterinary faculty, University of Ljubljana, Slovenia), Lawrence Gavey (Animal Health Ireland, Ireland), Linda van Wuyckhuise (Royal GD, the Netherlands), Luigi Ruocco and Marco Ianniello (Ministero della Salute, Italy), Marek Lipiec and Jerzy Rola (National Veterinary Research Institute, Poland), Marius Masiulis, Vaidas Jonušaitis (State Food and Veterinary Service, Lithuania), and Rahel Struchen, Sara Schärrer and Heinzpeter Schwermer (Federal Food Safety and Veterinary Office, Switzerland) for helping to gather the data for this study.
The full questionnaire can be found in the website of the project: section of Reports and Publications [Reports & Publications - SOUND control - COST Action CA17110 (sound-control.eu)] or it can be requested from the ER.

Footnotes

**Funding.** This study was part of SOUND control COST Action (CA17110), funded by Horizon 2020 Framework Program of the European Union and COST—European Cooperation in Science & Technology.

References

1. World Trade Organisation (WTO). The WTO Agreement on the Application of Sanitary and Phytosanitary Measures (SPS Agreement). (1995). Available online at: www.wto.org/english/tratop_e/sps_e/spsagr_e.htm (accessed December 20, 2020).

2. World Organisation for Animal Health (OIE). Terrestrial Animal Health Code. (2019). Available online at: https://www.oie.int/index.php?id=169&L=0&htmfile= titre 1.5.htm (accessed December 17, 2020).

3. European Commission. Regulation (EU) 2016/429 of the European Parliament and of the Council of 9 March 2016 on transmissible animal diseases and amending and repealing certain acts in the area of animal health (“Animal Health Law”). Official J Europ Union. (2016). 208. Available online at: http://data.europa.eu/eli/reg/2016/429/oj

4. Thiermann AB. Globalization, international trade and animal health: the new roles of OIE. Prev Vet Med. (2005) 67:101–8.
10.1016/j.prevetmed.2004.11.009 [PubMed] [CrossRef] [Google Scholar]

5. World Organisation for Animal Health (OIE). OIE-Listed Diseases, Infections and Infestations in Force in 2021. (2021). Available online at: https://www.oie.int/animal-health-in-the-world/oie-listed-diseases-2021/ (accessed March 19, 2021).

6. Richter V, Kattwinkel E, Firth CL, Marschik T, Dangelmaier M,
Traffler M, et al.. Mapping the global prevalence of bovine viral diarrhoea virus infection and its associated mitigation programmes. Vet Rec. (2019) 184:711. 10.1136/vr.105354 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

7. van Roon AM, Santman-Berends IMGA, Graham D, More SJ, Nielen M, Madouasse A, et al.. STOC free: an innovative framework to compare probability of freedom from infection in heterogeneous control programmes. Front Vet Sci. (2019) 6:1–5. 10.3389/fvets.2019.00133 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

8. Küker S, Faverjon C, Furrer L, Berezowski J, Posthaus H, Rinaldi F, et al.. The value of necropsy reports for animal health surveillance. BMC Vet Res. (2018) 14:1–12. 10.1186/s12917-018-1505-1 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

9. Foddai A, Nielsen LR, Willeberg P, Alban L. Comparison of Output-Based Approaches Used to Substantiate Bovine Tuberculosis Free Status in DANISH CATTLE HERDS. Copenhagen: Elsevier BV. (2015). 10.1016/j.prevetmed.2015.05.005 [PubMed] [CrossRef] [Google Scholar]

10. Martin PAJ. Current value of historical and ongoing surveillance for disease freedom: Surveillance for bovine Johne's disease in Western Australia. Prev Vet Med. (2008) 84:291–309. 10.1016/j.prevetmed.2007.12.002 [PubMed] [CrossRef] [Google Scholar]

11. More SJ, Cameron AR, Greiner M, Clifton-Hadley RS, Rodeia SC, Bakker D, et al.. Defining output-based standards to achieve and maintain tuberculosis freedom in farmed deer, with reference to member states of the European Union. Prev Vet Med. (2009) 90:254–67. 10.1016/j.prevetmed.2009.03.013 [PubMed] [CrossRef] [Google Scholar]

12. Martin PAJ, Cameron AR, Barfod K, Sergeant ESG, Greiner M. Demonstrating freedom from disease using multiple complex data sources. 2: case study-classical swine fever in Denmark. Prev Vet Med. (2007) 79:98–115. 10.1016/j.prevetmed.2006.09.007 [PubMed] [CrossRef] [Google Scholar]

13. Schuppers ME, Stegeman JA, Kramps JA, Stärk KDC. Implementing a probabilistic definition of freedom from infection to facilitate trade of livestock : putting theory into praxis for the example of bovine herpes virus-1. Prev Vet Med. (2012) 105:195–201. 10.1016/j.prevetmed.2011.12.013 [PubMed] [CrossRef]
[Google Scholar]

14. Alban L, Boes J, Kreiner H. Towards a risk-based surveillance for Trichinella spp. in Danish pig production. Prev Vet Med. (2008) 87:340–57. 10.1016/j.prevetmed.2008.05.008 [PubMed] [CrossRef] [Google Scholar]

15. Cameron AR. The consequences of risk-based surveillance: developing output-based standards for surveillance to demonstrate freedom from disease. Prev Vet Med. (2012) 105:280–6. 10.1016/j.prevetmed.2012.01.009 [PubMed] [CrossRef] [Google Scholar]

16. Cameron SJ, Greiner M, Clifton-Hadley RS, Correia Rodeia S, Verloo D, Wierup MAR. Output-based standards in animal health: prevention of the spread of tuberculosis as an example. ISVEE Proc. (2009) 2–4. 10.2903/j.efsa.2008.645 [CrossRef] [Google Scholar]

17. Madouasse A, Mercat M, van Roon A, Graham D, Guelbenzu M, Berends IS. A modelling framework for the prediction of the herd-level probability of infection from longitudinal data. bioRxiv. (2020) 2020.07.10.197426. 10.1101/2020.07.10.197426 [CrossRef] [Google Scholar]

18. Martin PAJ, Cameron AR, Greiner M. Demonstrating freedom from disease using multiple complex data sources. 1: a new methodology based on scenario trees. Prev Vet Med. (2007). 79:71–97. 10.1016/j.prevetmed.2006.09.008 [PubMed] [CrossRef] [Google Scholar]

19. Emanuelson U, Egenvall A. The data - Sources and validation. Prev Vet Med. (2014) 113:298–303. 10.1016/j.prevetmed.2013.10.002 [PubMed] [CrossRef] [Google Scholar]

20. Peyre M, Hoinville L, Njoroge J, Cameron A, Traon D, Goutard F, et al.. The RISKSUR EVA tool (Survtool): A tool for the integrated evaluation of animal health surveillance systems. Prev Vet Med. (2019) 173:104777. 10.1016/j.prevetmed.2019.104777 [PubMed] [CrossRef] [Google Scholar]

21. Zancanaro G, Antoniou SE, Bedriova M, Boelaert F, Gonzales Rojas J, Monguidi M, et al.. SIGMA Animal Disease Data Model. EFSA J. (2019) 17:1–60. 10.2903/j.efsa.2019.5556 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

22. Memorandum of Understanding for the implementation of the COST Action . “Standardizing output-based surveillance to control non-regulated diseases of cattle in the EU” (SOUND-control) CA17110. Brussels: COST Association; (2018). Available online at: https://sound-control.eu/wp-content/uploads/2019/04/CA17110-
[e.pdf](#) [Google Scholar](#)

23. Costa L, Duarte EL, Knific T, Hodnik JJ, van Roon A, Fourichon C, et al.. Standardizing output-based surveillance to control non-regulated cattle diseases: aspiring for a single general regulatory framework in the European Union. Prev Vet Med. (2020) 183:105130. 10.1016/j.prevetmed.2020.105130 [PMC free article] [PubMed](#) [CrossRef](#) [Google Scholar](#)

24. van Roon AM, Rapaliute E, Koleci X, Muñoz V, Mercat M, Faverjon C, et al.. Key learnings during the development of a generic data collection tool to support assessment of freedom of infection in cattle herds. Front Vet Sci. (2021) 8:336. 10.3389/fvets.2021.656336 [PMC free article] [PubMed](#) [CrossRef](#) [Google Scholar](#)

25. Schmitz LPT/ C. LimeSurvey: An Open Source Survey Tool. Hamburg: (2012). Available online at: [http://www.limesurvey.org](http://www.limesurvey.org) [Google Scholar](#)

26. Microsoft Corporation. Microsoft Excel. Available online at: [https://office.microsoft.com/excel](https://office.microsoft.com/excel)

27. R Core Team. R: A Language and Environment for Statistical Computing. Austria: (2020). Available online at: [https://www.r-project.org/](https://www.r-project.org/) [Google Scholar](#)

28. Hadley Wickham KM, Romain François LH. dplyr: A Grammar of Data Manipulation. (2020). Available online at: [https://cran.r-project.org/package=dplyr](https://cran.r-project.org/package=dplyr)

29. Wickham H. ggplot2: Elegant Graphics for Data Analysis. (2016). Available online at: [https://ggplot2.tidyverse.org](https://ggplot2.tidyverse.org). 10.1007/978-3-319-24277-4 [CrossRef](#)

30. van Roon AM, Santman-Berends IMGA, Graham D, More SJ, Nielen M, van Duijn L, et al.. A description and qualitative comparison of the elements of heterogeneous bovine viral diarrhea control programs that influence confidence of freedom. J Dairy Sci. (2020) 103:4654-71. 10.3168/jds.2019-16915 [PubMed](#) [CrossRef](#) [Google Scholar](#)

31. European Commission. REGULATION (EC) No 1760/2000 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL establishing a system for the identification and registration of bovine animals and regarding the labelling of beef and beef products and repealing Council Regulation (EC) No 820/9. Official Journal of the European Union. (2010) 1-27. Available online at: [http://data.europa.eu/eli/reg/2000/1760/oj](http://data.europa.eu/eli/reg/2000/1760/oj)
32. European Commission. TRACES (TRAde Control and Expert System). Available online at: https://ec.europa.eu/food/animals/traces_en (accessed March 25, 2021).

33. Roon Van AM, Mercat M, Schaik Van G, Nielen M, Graham DA, More SJ, et al.. Quantification of risk factors for bovine viral diarrhea virus in cattle herds : a systematic search and meta-analysis of observational studies. J Dairy Sci. (2020) 103:9446-63. 10.3168/jds.2020-18193 [PubMed] [CrossRef] [Google Scholar]

34. Mcaloon CG, Doherty ML, Whyte P, More SJ, Grady LO, Citer L, et al.. Relative importance of herd-level risk factors for probability of infection with paratuberculosis in Irish dairy herds. J Dairy Sci. (2017) 100:9245-57. 10.3168/jds.2017-12985 [PubMed] [CrossRef] [Google Scholar]

35. Boelaert F, Speybroeck N, Kruif de A, Aerts M, Burzykowski T, Molenberghs G, et al.. Risk factors for bovine herpesvirus-1 seropositivity. Prev Vet Med. (2005) 69:285-95. 10.1016/j.prevetmed.2005.02.010 [PubMed] [CrossRef] [Google Scholar]

36. Cameron A. Manual of Basic Animal Disease Surveillance. Nairobi, KENYA: African Union - Interafrican Bureau for Animal Resources (AU-IBAR) (2012). 3-5. [Google Scholar]

37. Salman M. Animal disease surveillance and survey systems methods and applications. In: First. Salman, editor. Animal Disease Surveillance and Survey Systems Methods and Applications. Iowa, United states of America: Blackwell Publishing Professional; (2003). p. 135-145. 10.1002/9780470344866 [CrossRef] [Google Scholar]

Articles from Frontiers in Veterinary Science are provided here
courtesy of **Frontiers Media SA**

Other Formats

* PDF (2.9M)

Actions

• Cite

• Collections

Add to Collections
○ Create a new collection

○ Add to an existing collection

Name your collection:

Name must be less than characters

Choose a collection:

Unable to load your collection due to an error

[Please try again](#)

Add Cancel

## Share

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

Permalink

Copy

## RESOURCES

• Similar articles

• Cited by other articles

• Links to NCBI Databases

[x]

Cite

Copy [Download .nbib](#) [Download .nbib](#)

Format: AMA APA MLA NLM

Follow NCBI
<figure><img src="image_9.png" /></figure>
f
in
<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

Connect with NLM
<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>
<figure><img src="image_14.png" /></figure>

National Library of Medicine

8600 Rockville Pike

Bethesda, MD 20894

Web Policies

FOIA

HHS Vulnerability Disclosure

Help

Accessibility

Careers

• NLM

• NIH

• HHS

• USA.gov