J Orthop Sci (2013) 18:298–320
DOI 10.1007/s00776-012-0337-2

ORIGINAL ARTICLE

Validity and reliability of a self-administered foot evaluationquestionnaire (SAFE-Q)
Hisateru Niki • Shinobu Tatsunami • Naoki Haraguchi •Takafumi Aoki • Ryuzo Okuda • Yasunori Suda •Masato Takao • Yasuhito Tanaka

Received: 1 October 2012 / Accepted: 4 November 2012 / Published online: 9 January 2013
Ó The Japanese Orthopaedic Association 2012

Abstract
Background The Japanese Society for Surgery of theFoot (JSSF) is developing a QOL questionnaire instrument
for use in pathological conditions related to the foot andankle. The main body of the outcome instrument (the SelfAdministered Foot Evaluation Questionnaire, SAFE-Q
version 2) consists of 34 questionnaire items, which provide five subscale scores (1: Pain and Pain-Related; 2:
Physical Functioning and Daily Living; 3: Social Functioning; 4: Shoe-Related; and 5: General Health and Well-

Naoki Haraguchi is a member of the Clinical Outcomes Committee of
the Japanese Orthopaedic Association (JOA). All of the authors
belong to The Clinical Outcomes Committee of the Japanese Society
for Surgery of the Foot (JSSF). The authors prepared the article on
behalf of those committees.
H. Niki (&)
Department of Orthopaedic Surgery,
St. Marianna University School of Medicine,
2-16-1 Sugao, Miyamae-ku, Kawasaki,
Kanagawa 216-8511, Japane-mail: h2niki@marianna-u.ac.jp
S. Tatsunami
Unit of Medical Statistics, Faculty of Medical Education
and Culture, St. Marianna University School of Medicine,
2-16-1 Sugao, Miyamae-ku, Kawasaki,
Kanagawa 216-8511, Japan
N. Haraguchi
Department of Orthopaedic Surgery,Tokyo Metropolitan Police Hospital,
4-22-1, Nakano, Nakano-ku, Tokyo 164-8541, JapanT. AokiDepartment of Orthopaedic Surgery, Nippon Medical School
Musashikosugi Hospital, 1-396, Kosugi-cho, Nakahara-ku,
Kawasaki, Kanagawa 211-8533, Japan

123

Being). In addition, the instrument has nine optionalquestionnaire items that provide a Sports Activity subscalescore. The purpose of this study was to evaluate the testretest reliability of the SAFE-Q.
Patients and methods Version 2 of the SAFE-Q was
administered to 876 patients and 491 non-patients, and thetest-retest reliability was evaluated for 131 patients. In
addition, the SF-36 questionnaire and the JSSF Scale scoring
form were administered to all of the participants. Subscalescores were scaled such that the final sum of scores ranged
between zero (least healthy) to 100 (healthiest).
Results The intraclass correlation coefficients were larger
than 0.7 for all of the scores. The means of the five subscalescores were between 60 and 75. The five subscales easily
separated patients from non-patients. The coefficients forthe correlations of the subscale scores with the scores on
R. Okuda
Department of Orthopaedic Surgery,
Division of Vital Care and Reconstructive Medicine,
Osaka Medical College, 2-7 Daigaku-machi,
Takatsuki, Osaka 569-8686, Japan
Y. SudaDepartment of Orthopaedic Surgery,
School of Medicine, Keio University,
35 Shinano-machi, Shinjyuku-ku,
Tokyo 160-8582, Japan
M. Takao
Department of Orthopaedic Surgery,
Teikyo University School of Medicine,
2-11-1 Kaga, Itabashi-ku, Tokyo 173-8606, Japan
Y. Tanaka
Department of Orthopaedic Surgery, Nara Medical University,
840 Sijyo-cho, Kashihara, Nara 634-8522, JapanSelf-administered foot evaluation

the JSSF Scale and the SF-36 subscales were all highly
statistically significantly greater than zero (p \ 0.001). The
means for the five JSSF Scale classification groups fellwithin a relatively narrow range, indicating that the SAFEQ labels are sufficiently similar to permit their use for all of
the JSSF Scale classifications.
Conclusion The present study revealed that the test-retest
reliability is high for each subscale. Consequently, theSAFE-Q is valid and reliable. In the future, it will bebeneficial to test the responsiveness of the SAFE-Q.

Introduction
Patient-based outcome instruments, which are used tomeasure changes in health status over time, have become
increasingly popular. The four basic types of patient-based
outcome instruments are generic, disease-specific, regionspecific, and patient-specific. A region-specific instrument
contains items specific to only one body part and can beused with several different disease states affecting a specific region. The Japanese Society for Surgery of the Foot(JSSF) is developing a QOL questionnaire for use inindividuals with pathological conditions related to the footand ankle as a region-specific outcome instrument.
The questionnaire, named the Self-Administered Foot
Evaluation Questionnaire (hereafter referred to as the ‘‘SAFEQ’’) version 1, was subjected to through an initial field test [1],after which it was revised to a second version [2]. The main
body of the SAFE-Q version 2 consists of 34 questionnaire
items, providing five subscale scores (1: Pain and Pain-Related; 2: Physical Functioning and Daily Living; 3: Social
Functioning; 4: Shoe-Related; and 5: General Health andWell-Being). In addition, the instrument has nine optionalquestionnaire items that provide a Sports Activity score.
The SAFE-Q version 2 was subjected to a limited field
test. Tentative scores for the five subscales were comparedto their corresponding scales in the Short Form 36 Health
Survey, version 2.0 (SF-36) [3] and the JSSF Scale score[4, 5], and the results obtained were reasonable [1].
Therefore, based upon its favorable performance in theprevious field test [2], the JSSF decided to evaluate thesecond version of the SAFE-Q further by applying it to alarger sample of patients with foot and ankle disorders aswell as a control sample of healthy teenagers and adults.
Because the factor structure of the responses to the
instrument was valid in the former study, the primary aimof the present field survey was to evaluate the test-retest
reliability. A secondary aim was to test the influence of
background factors such as region-specific classification,
age group, and gender on the subscale scores. This reportprovides an analysis of the data gathered in this second
field test of the second version of the SAFE-Q.

299

Patients and methods
Study group
In the present field survey, the SAFE-Q version 2 wasadministered to 876 patients with pathological conditions
related to the foot and ankle. A total of 491 non-patients
consisting of healthy teenagers and adult volunteers werealso analyzed. Both patients and non-patients had beenregistered in a total of 99 institutions in Japan.
Although the SAFE-Q version 1 has already been presented in a previous article [1], we have provided theSAFE-Q version 2 in ‘‘Appendix 1’’ for the sake of reader
convenience. In addition, the manual for the SAFE-Q isshown in ‘‘Appendix 2.’’
Among the 876 patients, 131 of them with stable pathological symptoms attended the test-retest reliability evaluation. The same questionnaire form was answered bythese patients twice in succession. The interval between thefirst and second tests was a minimum of eight weeks.
When the test was first administered, an SF-36 questionnaire form was also answered by the subjects, and the JSSF
Scale scoring form was recorded by a physician.
Ethical issue
This study was approved by the Life Ethics Committee of
St. Marianna University School of Medicine in 2007 (no.
1192). The elongation of the research period until 2014 wasapproved in 2012.
Statistical analysis
EFA and CFA
An exploratory factor analysis (EFA) and confirmatory
factor analysis (CFA) were performed. These were done
to determine whether the factor structure was stable,
given that the patient population in this field test comprised a wide variety of pathologies. Response data fromthe patients during the first administration (but not theretest) of this field test were subjected to the same EFA
and CFA as used in the first field test of the second
version.
Computation of subscale scores
Subscale scores were computed for each of the five subscales. To compute the scores, for each subscale, theaverage non-missing values of items contributing to thesubscale were computed for each respondent. Prior toaveraging, VAS items were rescaled to conform to theranges of the categorical items. Averages were then

123

300

H. Niki et al.

Each subscale’s scores were subjected to a random-effects
linear regression with test-retest as a categorical predictor.
The intraclass correlation coefficient (ICC) was computed
as the index of reliability for each scale. Ninety-five percent confidence intervals (95 % CIs) for ICCs were computed by parametric bootstrapping [6] using 100 bootstrap
samples of patients with scores for the scale for both test
and retest administrations of the questionnaire.

gender. Patient groups in the JSSF Scale classification wereas follows: ankle and hindfoot, midfoot, hallux, lesser toe,and rheumatoid arthritis. Respondents were grouped by age
as follows: 16–39, 40–64, and 65 and older, inclusive. For thepatient groups and patient-age groups, each of the five subscales was assessed by means of one-way analysis of variance (ANOVA). Gender comparisons were made by meansof Student’s t test in each subgroup of patients classified bypatient group and age group. Dunnett’s multiple comparisons test was performed afterward to compare patientgroups. In order to stabilize the variances in the presence offloor and ceiling effects, the data were arcsine square-root
transformed prior to performing ANOVA or other tests.

Comparison with JSSF Scale scores

Patient versus non-patient comparison

Spearman’s rank correlation coefficients were computed
between the scores for each of the five SAFE-Q subscales
and the JSSF Scale scores (which were only taken frompatient responders during the first administration of thequestionnaire).

Scores for each of the five subscales were compared
between patients (first administration of the questionnaire)and non-patients by means of the Mann–Whitney test. Thisnonparametric test was used for this comparison due toconcern over the large proportion of ceiling responses in
the healthy group.

rescaled so that the final sum of scores ranged between zero(least healthy) and 100 (healthiest), inclusive.
Test-retest reliability

Comparison with SF-36 scores
Sports items
Spearman’s rank correlation coefficients were computed
between the scores for each of the five SAFE-Q subscales
and those for each of the eight SF-36 subscales. Scores foreach of the eight SF-36 subscales were computed using theJapanese norm-based scoring method as prescribed in thecommercial instrument’s documentation [3]. Again, QOL
scores were only taken from patient responders during thefirst administration of the questionnaire.
Comparison of scores for the Pain and Pain-Related
subscale and the SF-36 Bodily Pain subscale
We compared the patients’ scores for the Pain and PainRelated subscale with the scores for the SF-36 Bodily
Pain subscale. For this purpose, we extracted the values for
the Pain subscale scores from the JSSF Scale scores. On the
JSSF Pain subscale, 0, 20, 30, and 40 points are assigned topatients with diseases of the ankle and hindfoot, midfoot,hallux, and lesser toe, respectively; and 0, 10, 20, or 30 points
are assigned to patients with rheumatoid arthritis. Thus, we
computed the Spearman’s rank correlation coefficient between
the JSSF Pain score and the Pain and Pain-Related score or
SF-36 Bodily Pain score for each of the patient groups.
Background factors
The following patient characteristics were assessed usingscores from the first administration of the questionnaire:patient group in the JSSF Scale classification, age group, and123Sports-related questionnaire items were scored as above,taking into account the reversal of sense of the VAS item
among them. EFA was applied to the responses of patientsduring the first administration of the questionnaire in order
to confirm the unidimensionality of the scale. The testretest reliability of the sum of these items’ scores wasassessed as above.
Statistical probability
In the statistical comparisons, a p value of less than 0.05
was considered statistically significant. Below, for all
p values less than 0.001, we simply state p \ 0.001, evenwhen the exact value was obtained from the computation.

Results
Patient and non-patient classification and age
The classification of the subjects enrolled in the present
field study is summarized in Table 1. A total of 876
patients and 491 non-patients were registered. The majorityof the patients had diseases of the ankle and hindfoot (469).
Numbers of patients in the lesser toe (45) and midfoot (68)
groups were less than 100. The JSSF region-specific classification was not reported for eight patients. The mean ageof the patients in each group and that of the non-patientsSelf-administered foot evaluation

301

Table 1 Numbers of patients and non-patients
Patient-group by JSSF
Scale classification

GenderNumberAge
Mean ± SD

Male

232

47.0 ± 18.6

Female237

52.6 ± 18.1

Total

469

49.8 ± 18.5

Patient
Ankle and hindfootHalluxLesser toe

Midfoot

Rheumatoid arthritis

Not reported

All groups

Male

43

54.9 ± 19.1

Female12659.5 ± 15.6

Total
Male

169
15

58.3 ± 16.6
43.7 ± 19.2

Female3052.7 ± 17.4

Total

45

49.7 ± 18.3

Male

32

44.8 ± 17.3

Female3650.7 ± 19.8

Total

68

47.9 ± 18.7

Male

30

61.0 ± 14.3

Female8759.3 ± 12.1

Total

117

59.8 ± 12.6

Male

4

31.5 ± 16.9

Female467.8 ± 7.5

Total

8

49.6 ± 22.8

Male

356

48.6 ± 18.8

Female520

55.4 ± 17.0

Total

876

52.6 ± 18.0

Non-patient
Male

225

45.0 ± 17.0

Female26644.2 ± 16.4

Total

491

44.6 ± 16.6

are also indicated in Table 1. As a whole, the mean ages ofthe patients and non-patients were 52.6 ± 18.0 (mean ± SD;
n = 876) and 44.6 ± 16.6 (n = 491), respectively.
Factor analysis
The factor structure was remarkably stable, in that factor
loadings and residual variances were essentially the sameas those obtained in the previous field test of the SAFE-Q
version 2 (data not shown). The factor correlation coefficients resulting from the CFA are summarized in Table 2.
All of the correlations between different subscale factors
were less than 0.9. The maximum coefficient was 0.841,
for the correlation between the Physical Functioning andDaily Living subscale and the Social Functioning subscale.
Test-retest reliability
The value of the ICC for each of the five subscales is listed
in Table 3. The ICC was always larger than 0.7; even theminimum 95 % CI lower limit for the Social Functioning
subscale was larger than 0.6. The ICC for the sum of thesubscale scores was 0.85 (with a 95 % CI of 0.81 to 0.89),
which is, as expected, higher than any of the individualcomponents.
Distribution of subscale scores
The distributions of the subscale scores are illustrated inFig. 1. The mean ± SD and median for the five subscales
were as follows: Pain and Pain-Related: 66.0 ± 23.8, 70.1;
Physical Functioning and Daily Living: 69.2 ± 26.2, 75.0;
Social Functioning: 66.3 ± 32.4, 75.0; Shoe-Related:
62.7 ± 30.4, 66.7; General Health and Well-Being:
66.8 ± 29.7, 75.0. The width between the 25th percentile
and the 75th percentile was broad in the Social Functioning, Shoe-Related, and General Health and Well-Being
subscales, while smaller widths were observed in the Pain
and Pain-Related and Physical Functioning and Daily
Living subscales. The values of the means were very
similar for the five subscales, ranging from 60 to 70.
Comparison with the JSSF Scale score
The distribution of the JSSF Scale score is illustrated inFig. 2. The mean ± SD and median were 69.4 ± 20.9 and
72 (n = 864), respectively. The JSSF score was correlatedwith each of the present subscale scores. The Spearman’srank correlation coefficients are summarized in Table 4,
where the patients are classified into JSSF patient groups.
The scores for the five subscales display statistically significant correlations (p \ 0.001) with the JSSF Scale score,with rank correlation coefficients ranging from 0.51 to 0.61
(Table 4). This tendency was the same in each group ofpatents. However, slightly smaller correlation coefficientswere observed in the lesser toe group containing 45
patients.
SF-36
The Spearman rank correlation coefficients between eachof the five subscales of the SAFE-Q and each of the eight
SF-36 subscales were all statistically significantly different
from zero (p \ 0.001), as summarized in Table 5. The
correlation coefficient for the Pain and Pain-Related subscale was highest with Bodily Pain; the correlation coefficient for the Shoe-Related subscale was highest with
Bodily Pain and Physical Functioning; the correlation
coefficient for the Physical Functioning and Daily Living
subscale was highest with Physical Functioning; the correlation coefficient for the Social Functioning subscale washighest with Role Physical and Bodily Pain (but nearly ashigh with Social Functioning and Physical Functioning);

123

302

H. Niki et al.

Table 2 Factor correlation coefficients among five subscales resulting from confirmatory factor analysis
Pain and Pain-Related(Q1–Q7, Q10, Q11)aPain and Pain-Related

Physical Functioning andDaily Living (Q12–Q22)

SocialFunctioning(Q23–Q28)

Shoe-Related(Q8, Q9, Q34)

GeneralHealth andWell-Being(Q29–Q33)

0.752 (0.016b)

0.647 (0.021)

0.721 (0.022)

0.785 (0.015)

0.841 (0.012)

0.737 (0.021)

0.808 (0.013)

Physical Functioning
and Daily Living
Social Functioning

0.726 (0.023)

Shoe-Relateda

Q1–Q34 refers to question item numbers (used in ‘‘Appendix 1’’) included in the corresponding subscalesbValue in parentheses is the standard error in the factor correlation coefficient

Table 3 Values of ICC observed for the five subscales
Subscale

0.804 (0.014)
0.718 (0.022)

ICC

95 % CI

Pain and Pain-Related

0.78

0.74–0.83

Physical Functioning and Daily Living

0.83

0.77–0.89

Social Functioning

0.72

0.64–0.79

Shoe-Related

0.81

0.76–0.86

General Health and Well-Being

0.82

0.78–0.87

Sum of scores

0.85

0.81–0.89

Comparison of scores from the SAFE-Q Pain and PainRelated subscale and SF-36 Bodily Pain subscale scores
Results of comparisons of the Spearman rank correlation
coefficients are summarized in Table 6. The Spearman’srank correlation coefficients from the Pain and Pain-Related subscale were larger than those from the SF-36 Bodily
Pain subscale in all groups of patients. Statistical significance (p \ 0.05) was found in the ankle and hindfoot andthe hallux groups.
Patient characteristics

the correlation coefficient for the General Health and WellBeing subscale was highest with Bodily Pain. In theseparticular patients, the scores obtained with these two
instruments were largely driven by pain and difficulty withmobility. The mean ± SD of each norm-based [3] SF-36
subscale score are also shown in Table 5. The mean of thenorm-based SF-36 score ranged from 36 to 47 for thesepatients, indicating that the patients were somewhat below
average in their health status.

Fig. 1 Subscale score
distributions. Left and rightrectangle edges indicate the25th and 75th percentiles.
Vertical lines within therectangles show the medians.
Bullet marks indicate the means.
Left and right ends of thehorizontal lines passing throughthe rectangles represent the 5th
and 95th percentiles

123

Comparison among patient groups
A comparison of the mean subscale scores and SDs of thedifferent JSSF patient groups is provided in Fig. 3. The
scores for the five patient groups were statistically significantly different according to one-way ANOVA, for all
subscales. The p values from ANOVA were smaller than
0.001 for the Physical Functioning and Daily Living andSelf-administered foot evaluation

303

Fig. 2 Distribution of the JSSF Scale scores for the present patientsthe Shoe-Related subscales, and were between 0.002 and0.02 for the other subscales. Patients with rheumatoid
arthritis showed the lowest mean values for the five subscales, and the differences between these mean values andthose of other patient groups were sometimes found to bestatistically significant upon performing Dunnett-typecomparison tests, as shown in Fig. 3.
Age and gender
The subscale scores for male and female patients werecompared for three age groups (ages 16–39, ages 40–64,

ages 65 and older, inclusive) in Fig. 4. The size of thesample analyzed in this work is large enough to allow
subscale-specific comparisons of scores among age groups
and genders. One-way ANOVA revealed that there were
statistically significant differences (p \ 0.001) among theage groups in all five subscales when only female patientswere considered. When only male patients were considered, there were no statistical significant differences among
the age groups for any of the subscales aside from the
Physical Functioning and Daily Life subscale (p \ 0.001).
The scores of male and female patients are also comparedin Fig. 4. In all subscales, the male scores were always
higher than the female scores, whichever age group wasconsidered; the differences between the male and female
scores were sometimes significant, as shown in Fig. 4.
Comparison of the scores of patients and non-patientsAs expected, patients scored lower (less healthy) on average than non-patients on each of the five subscales(Table 7). The p value from the Mann–Whitney testcomparing patients and non-patients was less than 0.001
for all five subscales. The means and SDs of the fivesubscale scores for non-patients are summarized inTable 8. Older non-patients tended to present lower meanvalues than younger non-patients, and female non-patients

Table 4 Correlations with the JSSF score for the five patient groups
Patient group by JSSF
Scale classificationn

Pain andPain-Related

Physical Functioning
and Daily Living

SocialFunctioning

Shoe-Related

General Healthand Well-Being

Ankle and hindfoot467

0.63 (p \ 0.001)

0.65 (p \ 0.001)

0.57 (p \ 0.001)

0.49 (p \ 0.001)

0.61 (P \ 0.001)

Hallux1670.64 (p \ 0.001)

0.50 (p \ 0.001)

0.50 (p \ 0.001)

0.46 (p \ 0.001)

0.52 (p \ 0.001)

Lesser toe

45

0.47 (p = 0.002)

0.52 (p \ 0.001)

0.51 (p = 0.001)

0.48 (p = 0.002)

0.45 (p = 0.004)

Mid foot68

0.61 (p \ 0.001)

0.69 (p \ 0.001)

0.63 (p \ 0.001)

0.54 (p \ 0.001)

0.58 (p \ 0.001)

Rheumatoid arthritis

117

0.57 (p \ 0.001)

0.64 (p \ 0.001)

0.59 (p \ 0.001)

0.55 (p \ 0.001)

0.56 (p \ 0.001)

All

864

0.61 (p \ 0.001)

0.60 (p \ 0.001)

0.55 (p \ 0.001)

0.51 (p \ 0.001)

0.56 (p \ 0.001)

Table 5 Comparison of scores for subscales of the SAFE-Q version 2 with SF-36 subscale scores
SF-36 subscale (mean ± SD)

SAFE-Q subscale
Pain andPain-Related

Physical Functioning
and Daily Living

SocialFunctioning

Shoe-Related

General Healthand Well-Being

Physical functioning (36.2 ± 18.4)

0.505 (p \ 0.001)

0.771 (p \ 0.001)

0.657 (p \ 0.001)

0.520 (p \ 0.001)

0.638 (p \ 0.001)

Role physical (36.9 ± 17.1)

0.422 (p \ 0.001)

0.625 (p \ 0.001)

0.704 (p \ 0.001)

0.436 (p \ 0.001)

0.607 (p \ 0.001)

Bodily pain (42.2 ± 11.8)

0.652 (p \ 0.001)

0.634 (p \ 0.001)

0.684 (p \ 0.001)

0.532 (p \ 0.001)

0.669 (p \ 0.001)

Social Functioning (46.2 ± 14.9)

0.406 (p \ 0.001)

0.579 (p \ 0.001)

0.667 (p \ 0.001)

0.446 (p \ 0.001)

0.592 (p \ 0.001)

General health (46.8 ± 11.5)

0.403 (p \ 0.001)

0.461 (p \ 0.001)

0.409 (p \ 0.001)

0.399 (p \ 0.001)

0.504 (p \ 0.001)

Vitality (47.3 ± 10.5)

0.415 (p \ 0.001)

0.400 (p \ 0.001)

0.450 (p \ 0.001)

0.375 (p \ 0.001)

0.507 (p \ 0.001)

Role emotional (42.3 ± 15.5)

0.442 (p \ 0.001)

0.543 (p \ 0.001)

0.600 (p \ 0.001)

0.416 (p \ 0.001)

0.610 (p \ 0.001)

Mental health (47.7 ± 11.0)

0.408 (p \ 0.001)

0.441 (p \ 0.001)

0.484 (p \ 0.001)

0.380 (p \ 0.001)

0.566 (p \ 0.001)

123

304

H. Niki et al.

tended to present lower mean values than male nonpatients.

was 45.3 ± 34.2 in patients, and it was 95.7 ± 10.9 in
non-patients. The difference in the mean scores of patientsand non-patients was statistically significant (p \ 0.001).

Sports items
Optional sports items were responded to by 275 patientsand 197 non-patients. EFA of the resulting patient data
showed that these items contributed to a single major
factor, as seen before (data not shown). The test-retest
reliability for sports items was similar to that observed forthe other sets of items: ICC = 0.76, with a 95 % CI of
0.64–0.87. The mean ± SD of the Sports Activity score

Table 6 Comparisons of Spearman’s rank correlation coefficients
between the present Pain and Pain-Related subscale score and the SF36 Bodily Pain subscale score
JSSF Scaleclassification

Rank correlation coefficients in comparisons of Painscores by JSSF Scales
SAFE-Q Pain andPain-Related

SF-36
Bodily Pain

SignificanceAnkle andhindfoot

0.63 (n = 409)

0.51 (n = 399)

p \ 0.05

HalluxLessor toe

0.68 (n = 163)
0.58 (n = 44)

0.47 (n = 160)
0.32 (n = 44)

p \ 0.01
NS

Midfoot

0.66 (n = 68)

0.50 (n = 67)

NS

Rheumatoid
arthritis

0.62 (n = 116)

0.47 (n = 113)

NS

Fig. 3 Comparison of themeans and SDs of the fivesubscale scores among the five
JSSF patient groups: 1 ankle
and hindfoot; 2 midfoot; 3
hallux; 4 lesser toe; and 5
rheumatoid arthritis. Asterisks(*p \ 0.05; **p \ 0.01)
indicate p values from Dunnetttype comparisons with therheumatoid arthritis group

Discussion
Several patient-based and region-specific outcome instruments for patients with diseases or injuries of the foot andankle region, such as the American Academy of Orthopaedic Surgeons lower limb outcomes assessment instruments (including the Foot and Ankle Module (AAOS-FA)[7], Foot and Ankle Ability Measure (FAAM) [8], Foot
Health Status Questionnaire (FHSQ) [9], and Foot Function
Index [10]), have been developed. Recently, a comparisonof the responsiveness of the Manchester–Oxford footquestionnaire (MOXFQ) with those of the American
Orthopaedic Foot Ankle Society [AOFAS] [11], SF-36[12], and EuroQol (EQ-5D) [13] assessments following
foot or ankle surgery was published [14]. Although the
MOXFQ is a patient-based outcome measure, it was originally developed based on interviews with patients who hadfoot surgery. In the interviews, however, the Manchester
Foot Pain and Disability Questionnaire (MFPDQ) [15] had
been utilized as a template. In addition, the measurement
properties of the MOXFQ were initially assessed in a
specific group of patients undergoing surgery for hallux
valgus [16, 17]. In this context, there is no new and original
patient-based outcome instrument focusing only on the footScore
0

Pain and Pain-Related

Physical Functioning
and Daily Living

1
2
3
4
5

20

40

60

80

100

**
**

1
2
3
4
5

Social Functioning

1
2
3
4
5
1
2
3
4
5

**

Shoe-Related

1
2
3
4
5

*

General Health andWell-Being

1: Ankle-hindfoot; 2: Midfoot; 3: Hallux; 4: Lesser Toes; 5: Rheumatoid Arthritis

123

**
**
**
**

**
*

**
**

Self-administered foot evaluation

305

Fig. 4 Mean subscale scores(and their SDs) for each age
group (1 ages 16–39, 2 ages
40–64, 3 ages 65 and older,
inclusive) and gender (opencolumn male; closed column
female). **p \ 0.01 forcomparisons between genders.
Using ANOVA, the female-onlyscores were found to besignificantly different among all
age groups and subscales(p \ 0.001), but when only
males were considered, only the
Physical Functioning and Daily
Living subscale scores were
only significantly different
among the age groups

Subscale Score
0

20

40

60

80

100

1

Pain and Pain-Related

2
3

**

1

Physical Functioning and2
Daily Living
3

**

1
2

Social Functioning

3

**

1

Shoe-Related

**

2

**
**

3

1

General Health andWell-Being

2
3

**

Table 7 Comparison of the subscale scores of patients and non-patients
Subscale

Patients

Non-patients

25th percentile

Median75th percentile

25th percentile

MedianComparison between
patients and non-patients

Pain and Pain-Related

47

70

84

94

100

p \ 0.001

Physical Functioning and Daily Living

55

75

92

98

100

p \ 0.001

Social Functioning

42

75

96

100

100

p \ 0.001

Shoe-Related

42

67

92

92

100

p \ 0.001

General Health and Well-Being

45

75

90

100

100

p \ 0.001

Sports Activity

11

34

75

97

100

p \ 0.001

Table 8 Means and SDs of the five subscale scores for non-patients, classified by age and genderGroup
Age groupa

Gendera

Pain and Pain-Related

Physical Functioning
and Daily Living

Social Functioning

Shoe-Related

General Healthand Well-Being1

96 ± 7 (n = 224)

99 ± 3 (n = 224)

99 ± 5 (n = 224)

94 ± 12 (n = 224)

99 ± 3 (n = 224)2

95 ± 10 (n = 178)98 ± 6 (n = 178)99 ± 7 (n = 177)92 ± 13 (n = 178)99 ± 5 (n = 178)3

94 ± 12 (n = 89)

93 ± 12 (n = 89)

95 ± 12 (n = 88)

91 ± 14 (n = 88)

94 ± 14 (n = 89)Male

97 ± 7 (n = 225)

98 ± 6 (n = 241)99 ± 6 (n = 224)

97 ± 7 (n = 224)

98 ± 7 (n = 225)

Female94 ± 11 (n = 266)

97 ± 7 (n = 266)

98 ± 9 (n = 265)89 ± 15 (n = 266)

98 ± 7 (n = 266)

1 16–39 years old, 2 40–64 years old, 3 65–88 years old

and ankle that is similar to the various instruments thathave already been verified to be valid, repeatable, andreliable.

There are potential advantages and disadvantages associated with each of these instruments [18], and there is an
ongoing process whereby evidence is collected to support

123

306

their use under various conditions. The usefulness of an
outcome instrument is never completely established. Thereis currently an urgent need for scientific evaluation of footand ankle surgery, which in turn requires the use of
appropriate (patient-based) standard methods of outcome
assessment. In this context, the Japanese Society for Surgery of the Foot (JSSF) is developing a QOL questionnaire
for use in individuals with pathological conditions related
to the foot and ankle as a region-specific outcome
instrument.
The present field test of the second version of theSAFE-Q replicated the factor structure of the same version of the SAFE-Q in its first field test (which had asmaller patient sample). The test-retest reliability washigh for each of the subscales and for the average of
all subscales. Gender-related differences, observed inparticular for the Shoe-Related subscale and Physical
Functioning and Daily Living subscale, might reflect thewell-known foot-health consequences of women wearing
high-heeled footwear and women’s more fashion-oriented
attitude towards shoes. It is believed by many surgeons
that age-related differences reflect a general decline inoverall health and physical vigor, as well as a general
reduced ability to recover quickly from health-related
problems.
The differences between patient groups were also statistically significant according to ANOVA. In particular,
patients with rheumatoid arthritis appeared to fare morepoorly than patients in other region-specific categories(Fig. 3). Nevertheless, the averages for the patient groups
fell in a relatively narrow range, indicating that the SAFEQ labels are sufficiently similar to allow their use in all
patient groups.
As expected, the SAFE-Q readily distinguished patientswith foot and ankle disorders from non-patients. The meanscores on the subscales range between 60 and 75, which
may lead to concern over the sensitivity or dynamic rangeof the QOL instrument in these patients. In contrast to this,the distribution of JSSF scores observed in the patients
implies that most of the patients did not have severe
symptoms (Fig. 2). This is a plausible reason for the
scattered range of mean values observed in the present field
test.
Given the large sample size, the coefficients for thecorrelation of the SAFE-Q subscale scores with the JSSF
Scale score were all highly statistically significantly greaterthan zero. Likewise, the coefficients of the correlation of
the SAFE-Q subscale scores and SF-36 subscale scores
were statistically significantly greater than zero for the123

H. Niki et al.

same reason. Nevertheless, there was a qualitative alignment of the two QOL scales when the correlation coefficient values were examined. The lack of perfect alignment
indicates that the SAFE-Q constructs measured in thesepatients are superior to those measured by the corresponding subscales in the more general SF-36 instrument.
It does appear, however, that the scores obtained using both
instruments are largely driven by pain and difficulty withmobility in these patients.
The nine items of the Sports Activity subscale of theSAFE-Q consist of questions about very basic performanceof sports activities [8, 19, 20]. Regarding the Sports
Activity subscale, the unidimensionality of the items
remained stable and the difference between patients andnon-patients was apparent. In addition, the test-retest reliability was adequate. Therefore, we will add these nine
items to the responsiveness analysis without changing
them.
As reviewed by Martin and Irrgang [18], validity testingof QOL outcome instruments should include assessmentsof content validity, construct validity, test-retest reliability,
and responsiveness. In our process, content validity wasconfirmed for the SAFE-Q version 1 [1] and version 2 [2]through the various Cronbach a metrics. Regarding construct validity, we ascertained convergence by comparing
the SAFE-Q subscales with the JSSF scales and SF-36
subscales. We also studied the convergence and divergence[21] by evaluating the results from CFA. That is, weobserved that the factor loading of each questionnaire item
was large for the intended subscale and small for the othersubscales in the previous field study, and similar results
were seen in the present study.
As described above, we were able to verify that the testretest reliability was high for each subscale. The comparison of Spearman rank correlation coefficients shown in
Table 6 suggests that the Pain and Pain-Related subscale ismore responsive than the SF-36 Bodily Pain subscale.
However, there is no other clear standard that could be
used to gauge the responsiveness of the other subscales.
Additionally, the responsiveness should be evaluated by
performing a longitudinal study. In the future, it will bebeneficial to test the responsiveness of the present outcome
instrument.
Acknowledgments This study was supported by grants from the
JSSF (Japanese Society for Surgery of the Foot) and JOA (Japanese
Orthopaedic Association). In addition, the authors would like to thankall of the orthopedic surgeons who collaborated with the field survey.
We declare that we have no conflict of interest regarding the present
manuscript.

Self-administered foot evaluation

307

Appendix 1

Self-Administered Foot Evaluation Questionnaire (SAFE-Q)
Sex
Patient Name

Date of Birth

1. Male2. Female(Month DD, YYYY)

Date When Answered(Month DD, YYYY)

ID Number:

The following questionnaire is intended to ask about the condition of your feet, and what causes
you difficulties and problems in your daily life. The questionnaire does not only contain questions
on pain and physical function, but also emotion-related questions. It also asks about how yourquality of life may be affected by foot disease and/or injury.
We believe that your honest opinion will benefit future foot treatment and footcare aids.
We would appreciate it if you would take the time to answer the questionnaire.
When we use the word “foot,” we here refer to the parts framed by the rectangle in the illustration
below, that is, the entire part from the shank through the top of the toes; the knees are notincluded.

The word “foot” in this questionnairerefers to the parts framed by therectangle in the left illustration. The
knees are not included.

= = = = = = = Precautions When Filling out the Questionnaire = = = = = = =[1] Please think back about the last week or month, and then answer the questions.
[2] Each question also gives you an explanation on how to answer the question; please read the
explanation carefully and give your answer. It will take about 10 minutes to fill out thequestionnaire although the time varies among individuals.
[3] There are two ways of answering the questions in this questionnaire.
I. Put a tick ( ) in the appropriate box.
II. Put a cross (×) on the line.

123

308

123

H. Niki et al.

Self-administered foot evaluation

309

123

310

123

H. Niki et al.

Self-administered foot evaluation

311

123

312

123

H. Niki et al.

Self-administered foot evaluation

313

Q

123

314

123

H. Niki et al.

Self-administered foot evaluation

315

123

316

Appendix 2

123

H. Niki et al.

Self-administered foot evaluation

317

123

318

H. Niki et al.

25
–
9

25
–
11

123

Self-administered foot evaluation

319

25
–
6

25
–
3

25
–
5

123

320

H. Niki et al.

25–
9References
1. Niki H, Tatsunami S, Haraguchi N, Aoki T, Okuda R, Suda Y,
Takao M, Tanaka Y. Development of the patient-based outcome
instrument for the foot and ankle. Part 1: project description andevaluation of the outcome instrument version 1. J Orthop Sci.
2011;16:536–55.
2. Niki H, Tatsunami S, Haraguchi N, Aoki T, Okuda R, Suda Y,
Takao M, Tanaka Y. Development of the patient-based outcome
instrument for the foot and ankle. Part 2: results from the second
field survey: validity of the outcome instrument for the foot andankle version 2. J Orthop Sci. 2011;16:556–64.
3. Fukuhara S, Suzukamo Y. Manual of SF-36v2 (Japanese version). Kyoto: Institute for Health Outcomes and Process Evaluation Research; 2004.
4. Niki H, Aoki H, Inokuchi S, Ozeki S, Kinoshita M, Kura H,
Tanaka Y, Noguchi M, Nomura S, Hatori M, Tatsunami S.
Development and reliability of a standard rating system for outcome measurement of foot and ankle disorders I: development ofstandard rating system. J Orthop Sci. 2005;10:457–65.
5. Niki H, Aoki H, Inokuchi S, Ozeki S, Kinoshita M, Kura H,
Tanaka Y, Noguchi M, Nomura S, Hatori M, Tatsunami S.
Development and reliability of a standard rating system for outcome measurement of foot and ankle disorders II: interclinician
and intraclinician reliability and validity of the newly establishedstandard rating scales and Japanese Orthopaedic Association
rating scale. J Orthop Sci. 2005;10:466–74.
6. Park HM. Evaluating interrater agreement with intraclass correlation coefficient in SPICE-based software process assessment.
In: IEEE, editor. Proceedings of the Third International Conference on Quality Software. Washington, DC: IEEE; 2003.
p. 308–14.
7. Johanson NA, Liang MH, Daltroy L, Rudicel S, Richmond J.
American Academy of Orthopaedic Surgeons lower limb outcomes assessment instruments: reliability, validity, and sensitivity to change. J Bone Jt Surg Am. 2004;86:902–9.
8. Martin RL, Irrgang JJ, Burdett RG, Conti SF, Van Swearingen
JM. Evidence of validity for the foot and ankle ability measure(FAAM). Foot Ankle Int. 2005;26:968–83.
9. Bennett PJ, Patterson C, Wearing S, Baglioni T. Development
and validation of a questionnaire designed to measure foot-healthstatus. J Am Podiatr Med Assoc. 1998;88:419–28.

123

10. Budiman-Mak E, Conrad KJ, Roach KE. The foot function index:
a measure of foot pain and disability. J Clin Epidemiol.
1991;44:561–70.
11. Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS,
Sanders M. Clinical rating systems for ankle-hindfoot, midfoot,hallux and lesser toes. Foot Ankle Int. 1994;15:349–53.
12. Ware JE, Sherbourne CD. 36-Item Short-Form Health Survey(SF-36). I. Conceptual framework and item selection. Med Care.
1992;30:474–83.
13. Group EuroQol. EuroQol; a new facility for the measurement ofhealth-related quality of life. Health Policy. 1990;16:199–208.
14. Dawson J, Boller I, Doll H, Lavis G, Sharp R, Cooke P, Jenkinson C. Responsiveness of the Manchester-Oxford foot questionnaire (MOXFQ) compared with AOFAS, SF-36 and EQ-5D
assessments following foot or ankle surgery. J Bone Jt Surg Br.
2012;94:215–21.
15. Garrow AP, Papageorgiou AC, Silman AJ, Thomas E, Jayson MI,
Macfarlane GJ. Development and validation of a questionnaire toassess disabling foot pain. Pain. 2000;85:107–13.
16. Dawson J, Coffey J, Doll H, Lavis G, Cooke P, Herron M, Jenkinson C. A patient-based questionnaire to assess outcomes offoot surgery: validation in the context of surgery for hallux valgus. Qual Life Res. 2006;15:1211–22.
17. Dawson J, Doll H, Coffey J, Jenkinson C. Responsiveness andminimally important change for the Manchester-Oxford footquestionnaire (MOXFQ) compared with AOFAS and SF-36
assessments following surgery for hallux valgus. Osteoarthr
Cartil. 2007;15:918–31.
18. Martin RL, Irrgang JJ. A survey of self-reported outcome
instruments for the foot and ankle. J Orthop Sports Phys Ther.
2007;37:72–84.
19. Seligson D, Gassman J, Pope M. Ankle instability: evaluation of
the lateral ligaments. Am J Sports Med. 1980;8:39–42.
20. Williams GN, Molloy JM, DeBerardino TM, Arciero RA, Taylor
DC. Evaluation of the sports ankle rating system in young, athletic, individuals with acute lateral ankle sprains. Foot Ankle Int.
2003;24:274–82.
21. Martin RL, Irrgang JJ, Lalonde KA, Conti S. Current concept
review: foot and ankle outcome instruments. Foot Ankle Int.
2006;27:383–90.