Comments on: Guidance for industry: Patient-reported outcome measures:
Use in medical product development to support labeling claims.

With great interest I have read the draft of the FDA guidance. This is
very important work. I have a few questions and suggestions, especially
refering to the evaluation of measurement properties of PRO instruments.
I hope these are useful to improve the guidance.

Kind regards,

Dr. Caroline Terwee

EMGO Institute,

VU University Medical Center,

van der Boechorststraat 7,

1081 BT Amsterdam,

the Netherlands

E-mail: cb.terwee@vumc.nl

Line 299, generation of items: What is considered an adequate number of
patients to support the opinion that the specific items in the
instrument are adequate and appropriate to measure the concept?

Table 4:

-   Why is the term reproducibility used in inter-interviewer
    reproducibility instead of inter-interviewer reliability? Is
    reliability considered synonymous to reproducibility or does this
    difference in terminology reflect a conceptual difference? If so,
    please specify.

-   It is not clear if all three mentioned aspects of validity should be
    assessed for all PRO instruments (e.g. should predictive validity be
    assessed for all PRO instrument?)

-   I like the use of the term "ability to detect change" in order to
    avoid confusion regarding the terminology of responsiveness or
    sensitivity to change.

-   Ability to detect change, what is assessed: Showing (by means of
    effect size statistics) that PRO scores have changed in the
    predicted direction when there has been a notable change in the
    patient is not enough. The real change in the concept that has
    occurred might be larger or smaller than the observed change, and
    the treatment effect might be under- or overestimated by the PRO
    instrument. I think a better criterion would be that changes in the
    PRO scores should correspond (not only in direction but also in
    magnitude) with changes in some external standard (anchor). This
    would also be more in line with the statement in lines 560-562, that
    a distribution-based approach alone is usually inadequate to derive
    an MID. I think the same applies to the ability to detect change.

-   Ability to detect change: I would suggest to add a standard under
    the heading "FDA review considerations" regarding the stability of
    PRO scores where there is no change (see "what is assessed"). For
    example: "Has the measurement error (e.g. standard error of
    measurement) been determined in patients similar to those
    participating in the clinical trial?"

-   Interpretability, test: ".... or, in some cases, any detectable
    difference". In which cases would that be?

-   I like the term "responder" to indicate important within-person
    change. However, it might be helpful to also refer to "minimal
    important change (MIC)" here, to emphasize (even more) the
    distinction between minimal important differences in changes between
    groups (MID) and minimal important within-person changes (MIC /
    responder). This difference is often not appreciated in the
    literature.

Line 520: When a concept is expected to change, the values for the PRO
instrument measuring that concept should change **to the expected
amount.**

In this context, I would suggest to put more emphasize on (assessing)
the validity of change scores of PRO instruments.

Line 526: The ability of an instrument to detect change indeed
influences the sample size needed. Therefore it should be demonstrated
that the instrument can detect an MID, with enough power, in a
reasonable (depending on population, treatment, etc.) sample size. Thus,
MID should be considered in relation to the measurement error (standard
error of measurement), e.g. by using a sample size fomula. This could be
emphasized more.

Line 551and 554: This should be about mapping **differences in changes
between groups** in PRO scores, instead of just "changes in PRO scores".

Line 554: mapping differences in changes between groups in PRO scores to
**changes in** other PRO scores ....

Line 550-564: Because the main interest with PRO instruments is in the
patient's opinion, I think that the patient qualities as the sole judge
of what is an important difference or change. Therefore, I think mapping
changes in PRO instruments to changes in non-PRO measures may not be an
adequate approach of assessing an MID. Furthermore, I would state that a
distribution-based approach is **always** inadequate and an anchor-based
approach (using the patient as the anchor) should be used instead. And
finally, the validity of the anchor should be justified.

Line 578: Review standards for defining responders are lacking.
Suggestions for appropriate methods could be:

-   Mapping change in PRO scores to a measure of global perceived
    effect;

-   Using consensus (e.g. patient focus groups) to define responders.

Line 631, changed mode of administration: proxy measures could be added
as an example.

Line 1024-1026: This section is unclear to me. Does the FDA suggests to
use MID in sample size calculations or not?

Line 1075: "The amount of difference or change..." this should be "the
amount of difference **in** change..."

Line 1079-1080: Are proxy measures considered to be non-PROs?

Line 1101-1112: Reliability and ability to detect change are not
mentioned here. Are these measurement properties considered not to be
part of the validation proces?

Line 1110: defining *a priori* hypotheses of the expected relationships
between PRO concepts and other measures\... This could be emphasized
more in section C2 (line 499-516). Few authors define and test specific
hypotheses for validity. Without specific hypotheses, the risk of biased
interpretation of the results is high.

Finally, I would like to call your attention to the COSMIN study
(COnsensus-based Standards for the selection of health Measurement
INstruments) <http://www.biomedcentral.com/1471-2288/6/2>, which is
currently being performed in an international Delphi panel. The results
of this study might be of interest to the FDA.