# Swiss Orthopaedics Minimal Dataset: First Pilot Report of Reliability and Validity

Authors: Thorsten Jentzsch; Claudio Dora; Urs Müller; Mazda Farshad
Journal: Advances in Orthopedics
 (2020)

---

## Abstract

Background. The Swiss Orthopaedics Minimal Dataset (SOMD) was launched seven years ago. It is a standardized, generic, and patient-reported outcome questionnaire, comprising ten items (location of disease, pain within the past four weeks, limitations at work/leisure/sleep/autonomy, subjective value of a body part, employment status, work disability (sick leave/pension), and household support). We conducted this study about the SOMD to report its reliability, validity, and clinical applicability. Methods. A retrospective observational cohort study was conducted. The test-retest study population (n = 60; lost to follow-up: n = 7 (12%)) was drawn from three retirement homes (in 2013), while the test study population (n = 14,180; excluded (e.g., duplicates): n = 1,990 (14%)) consisted of patients from a university hospital (in 2014–2017). In the test-retest study population, the same questionnaire was completed twice (at days 0 and 7). In the test study population, only the first questionnaire was included (to avoid duplicates). In a subgroup of the test study population (n = 302), only those patients who completed the SOMD and Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) of the hip within 14 days were considered (to minimize recall bias). Reliability (test-retest and internal consistency), criterion validity for the item of pain, and return rates were analyzed. Results. The test-retest study population (n = 53) showed very high test-retest reliability for all tested items of the SOMD (intraclass correlation coefficient = 0.96–1.00 (95% confidence interval 0.93–1.00), p<0.001). The test study population (n = 12,190) revealed good internal consistency reliability for all ten items (Cronbach’s alpha = 0.80). The return rates of the SOMD were improvable (43% in 2016 and 31% in 2017). The subgroup of the test study population (n = 302) displayed a borderline acceptable criterion validity (correlation of the item of pain between SOMD and WOMAC hip: rho = 0.62, p<0.001). Conclusion. This is the first report about the validation of the SOMD. A relatively high reliability (test-retest and internal consistency), borderline acceptable (criterion) validity for the item of pain, and improvable clinical implementation were observed. This analysis serves as the basis for a structured modification of the SOMD to improve its value.


---

## Body

## 1. Introduction

Health-related quality of life (HRQoL) describes the perceived well-being of individuals [1]. This can change over time (e.g., after treatment of disease). Accurate measurements (e.g., questionnaires) to detect this change are important not only for internal and external quality control but also for the evaluation of treatment success and costs. There are many questionnaires to choose from, but the gold standard remains elusive.The Swiss Orthopaedics Minimal Dataset (SOMD) was introduced seven years ago, in 2013, as a measuring tool for HRQoL in order to assess the indications and results of all orthopaedic surgeries. It is a standardized, generic, and patient-reported outcome questionnaire. It consists of ten items: location of disease (only at one body part), pain within the past four weeks (score of 0–100), limitations at work/leisure/sleep/autonomy (0–100 each), subjective value of a body part (0–100), employment status (training, employed, and retired), work disability ((A) sick leave and (B) pension) (0–100), and household support (0, <1, 1–2, 3–5, and >5 hours per day and a care assistant). The interpretation is based on a rating scale for each item without a single index value. It is free-of-charge, available in four languages (German, English, French, and Italian), accessible in electronic- and paper-based forms, and can be completed within five minutes. In our institution, it is used at first consultation and intermediate and final follow-ups.The psychometric properties (i.e., reliability, validity, and sensitivity) of the SOMD have not been adequately provided yet. So far, only content validity (i.e., expert opinion) has been established. The reliability, criterion validity, and clinical implementation (i.e., return rate) remain unknown. Reliability is defined as the consistency of an item. It can be assessed with a test-retest method providing an intraclass correlation coefficient (ICC) and internal consistency across different items within a test providing Cronbach’s alpha (α). Criterion validity describes the correlation between an item of a new questionnaire with the same item of an established questionnaire providing Spearman’s rho. A well-established, reliable, and valid questionnaire for comparison with the SOMD is the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), which was first described by Bellamy in a Master’s thesis in 1982 [2–7]. It is one of the most commonly used questionnaires for the hip, available in around 100 languages, and consists of 24 questions about pain, stiffness, and daily activities. The item of pain refers to the last two days and different situations (walking, stairs, bed, sitting, and standing), which is particularly detail-oriented. Scoring is performed on a five-point (or level) Likert scale from none, over, and moderate to extreme.We conducted the first study about the validation of the SOMD. The study hypothesis was that the SOMD is highly reliable, valid, and clinically applicable.


## 2. Methods

A retrospective observational cohort study was conducted. The cantonal ethics committee issued a waiver to allow this study with anonymous data without the need for informed consent (BASEC Request-Nr. 2018-00276).The test-retest study population (n = 60) was drawn from retirement homes in 2013. The loss to follow-up was acceptable (n = 7 (12%)). The retirement homes offered assisted but autonomous living. After thorough instruction of individuals, the German version of the SOMD was completed twice, initially at day 0 and again at day 7. Two items, employment status and work disability (sick leave/pension), were not evaluated in this elderly study population.The test study population (n = 14,180) consisted of all patients that filled out the questionnaire from a university hospital from April 2014 until December 2017. Only the first SOMD was considered. Duplicates and test questionnaires were excluded (n = 1,990 (14%)). Furthermore, a subgroup of the test study population was chosen, in which the WOMAC of the hip had also been completed within 14 days (to minimize recall bias) (n = 302).Data were given as medians (interquartile range (IQR)). For the test-retest study population, the test-retest reliability (ICC (95% confidence interval (CI))) was calculated. For the test study population, the internal consistency (Cronbach’sα of all ten items and item-rest correlations for each item), criterion validity (Spearman’s correlation (rho) of the item of pain in the SOMD and WOMAC of the hip), and return rates were calculated. A scatterplot is provided for illustration. The significance level was set at p<0.05. For a test-retest reliability test, it was previously suggested that a sample size of ≥46 individuals would be needed to measure an ICC of 0.9 with a power of 0.8 at a significance level of 5% [8]. For internal consistency, it was also suggested that a sample size of ≥300 individuals would be sufficient [9]. Both of our study populations surpassed these numbers. Stata (IC13.1; StataCorp, College Station, Texas, United States of America) was used for analysis.


## 3. Results

In the test-retest study population (n = 53), the median age was 75 (IQR 69–79) years. There were more females than males (females: n = 34 (57%) vs males: n = 26 (43%)). The scores for each item were moderate (e.g., pain: median = 50 (IQR 30–90)).The test-retest reliability for all tested separate items of the SOMD was very high (e.g., pain: ICC = 1.00 (95% CI 1.00–1.00),p<0.001) (Table 1). The best reliabilities were found for the items of pain, limitation at leisure and autonomy, subjective value of a body part, and household support (ICC = 1.00, each with CIs between 0.97 and 1.00).Table 1
Descriptive data and test-retest reliability for the test-retest sample (n = 60).
ItemTest (n = 60)Retest (n = 53)ICC (95% CI)P value∗Median (IQR)Median (IQR)Pain50 (30–90)60 (40–90)1.00 (1.00–1.00)<0.001Work limitation80 (25–100)80 (40–100)0.99 (0.99–1.00)<0.001Leisure limitation50 (15–100)60 (20–100)1.00 (1.00–1.00)<0.001Sleep limitation90 (25–100)90 (40–100)0.96 (0.93–0.98)<0.001Independence limitation90 (25–100)90 (50–100)1.00 (0.97–0.99)<0.001Subjective body part value70 (50–90)70 (50–90)1.00 (1.00–1.00)<0.001∗F-test. IQR: interquartile range; ICC: intraclass correlation coefficient; %: percent; CI: confidence interval. Note: interpretation of intraclass correlation coefficient (ICC): 0.00–0.19 = very weak; 0.20–0.39 = weak; 0.40–0.59 = moderate; 0.60–0.79 = high; 0.80–1.00 = very high.In the test study population (n = 12,190), the median age was 49 years (IQR 35–61). There were less females than males (females: n = 5,872 (48%) and males: n = 6,318 (52%)). The scores for each item were moderate (e.g., pain: median = 50 (IQR 30–70)) (Table 2). The right side was most commonly affected (right: n = 6,186 (51%); left: n = 4,615 (38%); axial: n = 1,376 (11%); not applicable: n = 13 (0%)). The shoulder, knee, and foot were most commonly impaired (shoulder: n = 2,390 (20%); knee: n = 2,348 (20%); foot: n = 1,969 (17%); pelvis: n = 1,393 (11%); axial: n = 1,376 (11%); hand: n = 664 (5%); ankle: n = 411 (3%); hip: n = 339 (3%); elbow: n = 253 (2%); thigh: n = 228 (2%); calf: n = 204 (2%); wrist: n = 359 (3%); upper arm: n = 156 (1%); lower arm: n = 87 (0%); not applicable: 13 (0%)).Table 2
Descriptive data for the test sample (n = 12,190).
ItemMedian (IQR)Pain50 (30–70)Work limitation50 (20–70)Leisure limitation70 (50–90)Sleep limitation30 (10–60)Independence limitation20 (0–50)Subjective body part value50 (30–70)Sick leave10 (10–10)IQR: interquartile range.The internal consistency for all ten items was very high (Cronbach’sα = 0.80). The item-rest correlations were the lowest for employment status (0.18), work disability (sick leave (0.40), pension (0.24)), and household support (0.36). After the removal of these items from the calculation of the internal consistency, Cronbach’s α increased to 0.85. The return rates of the SOMD were improvable (43% in 2016 and 31% in 2017).In the subgroup of the test study population (n = 302), the criterion validity for pain was borderline acceptable (correlation of the item of pain between the SOMD and WOMAC of the hip: rho = 0.62) (Figure 1).Figure 1
Item-rest correlations of all ten items of the test study population (n = 302).


## 4. Discussion

This is the first study reporting on the validation of the SOMD. Our findings show that the SOMD is highly reliable (i.e., very high test-retest reliability and internal consistency) but has borderline acceptable validity (i.e., correlation of the item of pain in the SOMD and WOMAC of the hip) and improvable clinical implementation (return rates).Overall, the SOMD appears as a valid measurement tool for the quantification of a joint-specific HRQoL. However, several modifications are necessary for revised future versions. In general, improvement of reliability can be achieved with item homogeneity (i.e., identical minimum and maximum of different items) and item selectivity (i.e., substantial differences in minimum and maximum of a single item). For improved internal and external quality control of surgical indications and outcomes, economic analyses, and research, a highly validated questionnaire, such as the EuroQol 5-Items (EQ-5D) [10], should be integrated into the SOMD to allow calculation of a single index value and quality-adjusted life years (QALYs). The EQ-5D is a standardized, generic, self-reported, fast, generalized, and validated questionnaire. It is available in 120 languages, highly reliable and valid, and commonly used to measure HRQoL. The first version with three levels was designed in 1990 and its current form with five levels (no, slight, moderate, severe, and extreme) has been used since 2009. It allows the calculation of 3,125 (55) different HRQoL states. Using crosswalk links, a single index value and QALYs (0 = dead to 1 = perfect health) can be calculated [11]. Using population value sets, this permits utility analyses. Furthermore, questions from the validated Core Outcome Measures Index (COMI) questionnaire [12] could be added to quantify the joint-specific HRQoL and level of pain. Additionally, survey reports can vary according to the educational level [13], which should be quantified by an additional question. We have proposed a revised version of the SOMD, which was drafted by a subcommittee of the Swiss Orthopaedics Panel of Quality and Methodology together with the general manager of the Swiss Implant Register (SIRIS) [14] and has been approved by Swiss Orthopaedics board of directors [15].


## 5. Conclusion

In conclusion, this is the first report about the validation of the SOMD. A relatively high reliability (test-retest and internal consistency), borderline acceptable (criterion) validity for the item of pain and improvable clinical implementation was observed. The analysis serves as the basis for a structured modification of an improved version of the SOMD.



---

*Source: 6673175-2020-12-30.xml*