Information extraction from full text scientific articles: Where are the keywords?

Background To date, many of the methods for information extraction of biological information from scientific articles are restricted to the abstract of the article. However, full text articles in electronic version, which offer larger sources of data, are currently available. Several questions arise as to whether the effort of scanning full text articles is worthy, or whether the information that can be extracted from the different sections of an article can be relevant.
Background
Most applications of information extraction from the scientific medical bibliography use the Abstract of the publication (for review see for example [1-3]). In the context of information extraction in molecular biology it is usually understood that the information to be extracted from an article are words regarding biological concepts that could synthesize the main points of the article (keywords). Therefore the Abstract of a paper is a good target for information extraction because by definition an abstract synthesizes the content of the article. Moreover, abstracts are available in public databases. However, nowadays most journals are also available in electronic version, and thus full text articles can be used for information extraction.
It is obvious that the full text of an article contains more information than its Abstract. However, in approaching full text analysis several problems must be tackled. On the one hand, the storage of full text articles requires more disk space and the analysis needs more computational capacity. On the other hand, an Abstract, as a summary, contains a high frequency of relevant terms (keywords), but this may not be the case of the rest of the article.
Other questions regard the quality of the information carried by different sections of an article. First of all, is the information in full text organized enough so that keywords can be extracted? Secondly, different biological concepts (for example, gene and protein names, tissue names, organisms, experimental conditions, etc.) may be located in different parts of the article. Or it could be that a word has a different meaning depending on the section where it is located (the word has a context dependent meaning). For example, regarding gene names, those found in the Methods section may refer mostly to analytical tools rather than being relevant to the biological phenomenology described in the whole article. In summary, it would be good to quantify and qualify the information in a full text article before embarking in large scale extraction of particular items of information.
With this goal in mind, we analyzed in this work the kind of information that is attached to different parts of an article and we tried to quantify how much information can be found in each section of an article. This should help to state some guidelines for researchers attempting to extract particular keywords (words synthesizing the content of the article) from full text articles.
Results
Text Corpus
As previously stated, the major objective of this work was to compare the information defined as keyword content carried by different sections of a paper, especially the differences between the Abstract and the rest. Therefore, as source for our analysis we used a set of full text articles with a regular section structure, in our study having a defined Abstract, Introduction, Methods, Results, and Discussion (A, I, M, R, D). Another requirement was certain homogeneity of style across the articles (for example, a similar length of the Methods section) and, since there is great interest in the field of data mining on the detection of gene names, the subject should be related to Genetics. Thus, we chose the 104 articles published in Nature Genetics from June 1998 (volume 19, issue 2) to June 2001 (volume 28, issue 2), which comply with the AIMRD structure. Note that other journals, or even the Letters of the very same Nature Genetics, might have a different structure (for example, lacking separated I, M, R, D sections).
Selection of Keywords
To simplify matters, and following our previous work [4], we focused on the extraction of relevant words (keywords) regarding objects, detected as nouns from natural text by a standard grammatical tagger (TreeTagger, Helmut Schmid, IMS, Stuttgart University, ). In order to derive keywords from the section of an article, we first compute the associations between the words in the section. Here, we take the sentence as the unit of text to look for associations, that is, two words are associated in the context of a section if they co-occur repeatedly in sentences within that section (see METHODS).
Since words associated strongly to many other words are relevant to the matter that is dealt in the article [5] we use a score (K) that is higher for words with many and strong relations to other words (see METHODS). This measure is used to select words as keywords, in this case, related to objects such as proteins, genes, organisms, etc.
In order to evaluate the performance of the keyword detection, we observed how the selected keywords matched the MeSH (Medical Subject Headings, ) terms attached by indexers at the National Library of Medicine to these 104 articles (18.6 on average). Since MeSH terms can be composed of several words (for example, "Learning Disorders"), we selected those composed of a single word (6.80 terms on average). We noted that the most unspecific (for example, animal) were often not present in the text and thus could not be matched by a keyword as opposed to species names (mouse, mycobacterium, human), or anatomical terms (hippocampus, cerebellum, breast). Of those single-word MeSH terms, 4.91 were found on average in the article (as nouns), and 2.22 were among the set of selected keywords (above K >= 0.3). Obviously, a more accurate comparison to MeSH terms would require the detection of bigrams, and trigrams (keywords composed of multiple words), but this is out the scope of our work. The recall when matching the original MeSH terms (6.80 on average) went down from 4.91 / 6.80 = 0.72 in the dictionary of 470.6 different nouns present in an article to 2.22 / 6.80 = 0.33 in the 66.6 keywords selected. However, since the size of the list of all nouns found in an article (470.6) is much larger than the number of keywords (66.6), the precision in matching the MeSH terms of an article increased from 4.91 / 470.6 = 0.010 to 2.22 / 66.6 = 0.033.
Keyword Selection by Section
The number of words selected upon a threshold in the K value varies for different sections (see Figure 1). The first observation is that there are a small number of words that have much better K scores than the rest. This means that the organization of words makes it possible to extract keywords for all the five considered sections.
Figure 1
Average number of keywords versus K for A, I, M, R, and D sections. The average number of nouns per section is, A = 52, I = 171, M = 404, R = 600, D = 331.
The number of selected words is very similar for all sections for very high values of K (above 0.8). Above a threshold on K (K >= 0.5; see Table 1) the resulting number of keywords is quite similar for Introduction and Methods (around 15 for each) with the other three sections producing around nine keywords. However, if one accounts for the size of the sections it is obvious that the frequency of keywords (selected with K >= 0.5) per noun is the best in the Abstract (0.18), followed by the Introduction (0.08), with Methods, Results, and Discussion lagging behind. This justifies data mining strategies that focus in the analysis of Abstracts in order to minimize computational resources. However, this result already indicates that not all keywords are in the Abstract, and that therefore mining the rest of the article may be worthy.
Table 1
Keyword selection per section.
all
K >= 0.3
K >= 0.4
K >= 0.5
A
52.17
19.44
14.42
9.77
I
171.32
31.03
20.47
14.00
M
404.19
54.24
28.50
15.80
R
599.98
24.74
12.74
7.85
D
331.04
26.16
14.25
8.75
Average number of nouns per section (all), or number of those selected as keywords for three different thresholds on the K score.
Sections Display Heterogeneous Information
As a way to show that the keyword content in different sections is heterogeneous, we examined which keywords (if any) were selected in all the sections of an article. Our results indicate that, as it could be expected, not many keywords are present in every section and those are not very relevant. Even for a low threshold of K >= 0.3, there is on average only one of such general keywords per article. Those are often non-informative words such as "gene", or "protein". This indicates that the information is unevenly distributed across the sections of the article, that is, different sections contain different kind of information.
We illustrate the heterogeneity of the information by section with the keywords selected (for K >= 0.5) for a particular article [6] (Figure 2). This work deals with a mutation of the Nf1 gene of mouse (an exon loss) that produces learning deficits. The only keyword present in every section is the organism under study, the mouse. If the Methods section is excluded, only one single more keyword (mutation) is selected. Other three-section overlaps give more interesting keywords such as the name of the gene under study (Nf1, neurofibromin), a domain contained in the resulting protein (GAP), the method for testing learning performance of mice (maze), or the resulting phenotype (impairment, lethality). Keywords unique to different sections tend to correspond to the different information contained in each section. For example, the keywords unique to the Methods section deal with reagents and techniques (antibody, amersham, tris, primer).
Figure 2
The keywords selected for an article [6] with a K >= 0.5 are represented as they appear in the different sections of the article.
In order to quantify the differences and similarities of content across the article we have used the number of keywords that are shared between different sections (Table 2). The values indicate that the Methods section is the most different of all. In Methods, the content is usually focused on the techniques and protocols used, and not so much on the biological phenomena that is the main subject of the article. This alone explains why those keywords present in every section (for example protein, gene) are scarce and uninteresting.
Table 2
Average number of keywords (K >= 0.5) shared by two sections for the corpus of 104 articles.
A
I
M
R
D
A
2.01
0.92
1.77
2.20
I
2.01
0.81
1.34
2.02
M
0.92
0.81
1.55
1.02
R
1.77
1.34
1.55
1.99
D
2.20
2.02
1.02
1.99
Regarding similarities between sections, A, I, and D are evenly similar among them, and R is the closest to M, as it is shown when plotting the distance matrix of Table 2 as a dendogram (see Figure 3). This is probably due to the fact that the Results section deals with the protocols used, although not as explicitly as the Methods section. The Discussion focuses again on the biological results (stressing their relation to the current knowledge) without detailing the techniques that have already been explained in Methods and justified in Results.
Figure 3
In order to display graphically the similarity between sections regarding keyword content, we took the inverse of the average number of shared keywords (Table 2) as a measure of dissimilarity between sections, and we plotted it as a dendogram (using [19]).
This result indicates that each section contains certain keywords that are unique to the section. In the following we try to characterize what are the differences in content between sections.
Qualitative Analysis of Subjects per Section
To make a deeper analysis of the kind of information present in each of the sections, we classified in seven categories a set of words present in our corpus of 104 articles (among the most frequent nouns). In order to do so as unambiguously as possible, we selected words that matched MeSH descriptors also consisting on that single word and belonging to only one major MeSH category (see METHODS). We added another category not present in MeSH, that of "Units, Dimensions, & Parts" in order to account for many terms that are currently not MeSH terms but are of interest to us.
The results (See Figure 4a) indicate that the large sections are a good source of keywords, obviously Methods gathering many terms related to techniques. Introduction, Results and Discussion contain a good deal of information regarding diseases. However, again, the Abstract section is shown as the best source for most subjects regarding frequency of keywords (Figure 4b) except for those typical of the Methods section (Techniques & Equipment; Chemicals & Drugs).
Figure 4
Word categories present in the five sections under analysis. Classes according to MeSH are A (Anatomy), B (Organisms), C (Diseases), D (Chemicals & Drugs), E (Techniques & Equipment), G (Biological Sciences). An additional class X was defined in this work (Units, Dimensions, & Parts). The number of words used for the analysis was 36 (class A), 14 (B), 11 (C), 47 (D), 33 (E), 41 (G), 49 (X). (a) Average number of occurrence of words of each subset per section. (b) Frequency of words of each subset per total number of words for each section.
Distribution of Gene Names
Since the detection of gene and protein names is a very important subject, broadly used for the detection of macromolecular interactions (see for example [7]), and because, as stated in the introduction, we are concerned about the relevance of matching gene names in different sections of an article, we examined the distribution of gene names across sections.
From a long list of genes names derived from the SWISSPROT database [8], we selected a very restricted set of 539 genes whose names are composed of three letters followed by one single digit, thus very difficult to be mistaken to other words not being genes. For example, there are gene names called Not or That. Shorter names (e.g. A6) can also be a problem. A total of 224 gene names out of the 539 was matched in 76 of the 104 articles. The Results section was the one with the greatest number of unique gene names (Figure 5a). Again, the Abstract, and then the Introduction, are the sections with the highest frequency of these names (Figure 5b).
Figure 5
Distribution of matches to a set of 224 gene names across sections. (a) Average number of unique gene names per section. (b) Frequency of different gene names per total of nouns for each section.
In order to illustrate the problems that affect gene-name identification if context is ignored (even using gene names apparently easy to recognize) (discussed for example in [9]) we checked manually the context of gene names that were exclusively mentioned in the Methods section. Of the 224 genes, just 24 were mentioned in the Methods section of the corresponding 14 articles and not elsewhere (see Table 3). In five of the 14 articles, the name was referring to a non-gene object (three restriction endonucleases, a vector name, and a fibroblast cell strain). In five articles, the gene was mentioned in a technical context (usually, the gene mRNA level was used for analysis of cell state) and no biological process involving the gene was described. In only five articles we found the mention of the gene name relevant (See Table 3). Additionally, we noted that of these 24 gene names, at least two (Pbp2, Pom1) could refer to two non-homologous (unrelated) genes, and another one (Sac1) to four; such polysemous gene names complicate gene identification from text. Biologists are aware of such problems (see for example [10]). In summary, extreme caution should be applied with gene names appearing uniquely in the Methods section because the context of gene names there is very different to that seen in the rest of the article. If automated methods to extract gene names from text are applied to the Methods section, those that explore the context of gene names using part-of-speech tagging (for example, [11]) or Hidden Markov Models (for example, [7]) should then perform better than those that just take co-occurrences of gene names [12,13].
Table 3
Detection of gene names appearing only in the Methods section.
Ref
Restriction endonucleases
Msp1
v27.n3.277
Incorrect:
Pst1
v19.n4.340
Sac1
v27.n4.375
Vector name
Psg5
v23.n3.287
Cell strain
Tig3
v26.n3.291
Definition of a Yeast strain
Can1, Leu2, Lys2, Trp1
v26.n4.415
In array
Faf1
v20.n3.266
Correct (technical context):
Growth detection
Mcm5, Mcm6
v25.n3.263
Platelet mRNA analysis
Pbp2
v23.n2.166
Primers used to determine embryo sex
Zfy1, Zfy2
v27.n1.31
Analysis of mutant phenotypes
Pmd1
v24.n4.355
cDNA probe
Rab2
v19.n2.134
Correct:
SNP found in cDNA
Add3, Npr2
v22.n3.239
Identifier given
Pom1
v28.n3.223
Detection of meiosis specific genes
Mei4, Mek1, Sps4, Zip1
v26.n4.415
Ref: reference of the article in Nature Genetics by volume, issue number and first page of the article.
Discussion
There is a clear need for doing information extraction of biological data from full text scientific articles and the means for doing it are there with computers better suited for faster computation every day and new methodologies for Natural Language Processing that can be used for biomedical literature (see for example [14]). Regarding the source of data, the full text electronic versions of journals are now more the rule than the exception, with initiatives in the way towards the construction of large public repositories of such information (although hotly debated; see about PubMed Central [15,16]).
In this work we have shown that the distribution of information (as keywords) in full text articles is heterogeneous and that there is certain correspondence of article sections with different kind and density of relevant data. The Abstracts are shown as the best repository from the point of view of having many keywords in a short space, justifying previous information extraction approaches. The lack of large repositories of full text articles in contrast to the current eleven million of references (many of them with their abstract) in the MEDLINE database, are another advantage of the Abstract approach.
However, we have shown that there is much more relevant information (at least in a ratio of 1:4 regarding gene names, anatomical terms, organism names, etc.) in the rest of the article. We have demonstrated that the information is structured enough to get important numbers of relevant keywords, but that for certain words (such as gene names) caution has to be taken regarding the context of the word.
We propose that the text mining of full text articles should be approached with different strategies for different sections. Beyond the Abstract, the Introduction looks like the best place to look for protein and gene names (and interactions) since it is probably describing current knowledge. The Discussion section, that interprets the results and put them in context with the current knowledge, looks like the third best place for mining such information, with Methods probably as the worst place. The Results section could be problematic given its mixed nature between Methods and the rest.
Regarding other subjects, such as keywords about biological concepts (species, tissues, diseases, etc.), again the Abstract and then the Introduction section look like the best sections to mine regarding frequency of such keywords, but Results and especially Discussion seem better from a quantitative point of view. The Methods section is clearly appropriated for looking for technical data, measurements, and chemicals. Respect to chemicals, again, their context can be completely different in this section compared to the rest.
Conclusions
Extraction of biological information from full text looks promising, but context must be regarded. Part of this context is given by the situation of the text under analysis within the article. Therefore, tuning the extraction of information to the section is probably a good strategy, and for particular tasks some sections should be avoided.
We have shown that the kind of simplistic annotation that constitutes tagging a fragment of an article as belonging to a characteristic section is already useful for text mining. But further tagging using markup codes in XML style [17] identifying biological objects and concepts (under development; see for example [18] or the GENIA project ) could ultimately make text mining a children's game. We hope for future interfaces for writers of Molecular Biology articles that should do the job upon validation by the authors (for example, marking every occurrence of a gene name with a unique and stable link to any of the existing gene sequence databases). For this to happen, the collaboration between both scientists and publishers will be very important.
Methods
Derivation of Associations between the words of a section
Given a section from an article, we split the text in sentences using a standard part of speech tagger (TreeTagger). We only computed associations between the words identified from the tagging as nouns. Following [4], the association between two words (wi,wj) (for example, "cell" and "cycle") can be modeled as the degree of inclusion of one word into the other () which can defined as the fuzzy binary relation given by: , that is, the ratio of the number sentences where both words wi and wj co-occur to the number of sentences the word wi occurs. This is an asymmetric relation very appropriate to model hierarchical relations between words as they happen in natural text. For example, in some Cell Biology context, the word "cycle" could appear always related to the word "cell" (as in "cell cycle"), but the word "cell" can be related to many other words such as in "cell growth", "cell membrane", or "cell nucleus". Accordingly, the inclusion value of the word "cycle" into "cell" will be close to one and the inclusion value of the word "cell" into the word "cycle" will be close to zero.
Selection of Keywords
We identify a word as relevant for the text analyzed if it establishes many and strong relations to other words (following [4]). Therefore, in a given section, we define a score for a word wi that is equal to , normalized to the maximum value found for K of any word in that section. Then, the keywords of the section are defined as those words that have a K score above a certain value.
Classification of Words in Subjects
In order to classify words into categories we used the following procedure. We chose the MeSH (Medical Subjects Headings) classification from the National Library of Medicine. All MeSH terms (including official synonyms) composed of one single word were selected and then the stem of the word was computed using TreeTagger. The words present in our corpus of 104 articles were ordered by frequency and all words occurring more than 200 times were selected. Those matching the selected single-word MeSH headers from six categories (A, B, C, D, E, and G; See the caption of Figure 4 for descriptions) were selected as belonging to those classes. In order to avoid possible miss-annotations, words matching more than one category were discarded. Manual analysis of the resulting table of associations was carried out in order to check the associations and to make new ones. A new class not present in MeSH (the X class of "Units, Dimensions, & Parts") was generated in order to include a large number of terms mainly present in the Methods section.
Authors' Contributions
PS carried out the analysis of the keyword distribution from a database of full text articles. CP developed and applied the method to compute keywords. MA prepared the figures (except fig 2 by PS) and conceptualised the structure of the paper. PB and MA co-directed the project and contributed to the final manuscript. All authors collaborated during the whole length of the project. All authors read and approved the final manuscript.

information extraction
keyword
full text article
gene name
data mining
text mining


Sick leave and work disability in patients with early arthritis

We studied the occurrence of sick leave and work disability, the presence of workplace adaptations and the usage of professional guidance related to working problems in patients with early arthritis. Inclusion criteria were arthritis symptoms of less than 2 years duration and a paid job at the time of diagnosis. Assessments were done in connection with an early arthritis clinic (EAC) at entry into the cohort and 12 months thereafter by means of a questionnaire comprising questions on sick leave (absenteeism from work reported to the employer), work disability (receiving a full or partial work disability pension), unemployment, work adaptations and professional guidance related to working problems. Fifty-seven of the 69 participants (83%) had an arthritis symptom duration of <6 months. The number of patients with sick leave due to arthritis in the past 12 months decreased from 28 (41%) at study entry to 18 (26%) after 12 months of follow-up. The number of patients receiving a work disability pension increased from 5 (7%) at study entry to 13 (19%) after 12 months of follow-up (10 partial and 3 full). Sick leave in the 12 months before study entry appeared to be the most important predictor of the institution or increase in a work disability pension (odds ratio, 16.1; 95%CI, 1.8–142.8). Between study entry and follow-up, the number of patients with workplace adaptations increased from 20 (29%) to 28 (42%), whereas the number of patients receiving vocational guidance decreased from 48 (70%) to 36 (52%). In patients with early arthritis and a paid job, arthritis-related sick leave was common and occurred in part before patients entered the EAC and a diagnosis was made. About 20% of the patients became permanently work disabled, with partial work disability being more common than full work disability. Considerable proportions of patients received workplace adaptations and professional guidance with working problems.
Introduction
Rheumatoid arthritis (RA) and other chronic inflammatory rheumatic diseases are disabling conditions that affect the lives of individual patients in many ways. The prevalence of work disability is high among patients with RA [1], as well as other forms of chronic arthritis, such as ankylosing spondylitis [2], systemic lupus erythematosus [3] and to a lesser extent psoriatic arthritis [4, 5]. In RA patients, it was found that most work disability develops rapidly after onset [6], with recent studies showing work disability rates among RA patients 1 year after diagnosis vary between 15 and 30% [7–9].
So far, research has focussed mainly on permanent work disability and its determinants [10]. Over the recent years, the interest in sick leave, which is considered to precede permanent work disability in chronic arthritis, is increasing. In cross-sectional studies in early RA (disease duration <2 years), the proportions of patients on sick leave varied from 5–55% [7, 11–13], whereas in follow-up studies, 50–60% of the patients reported sick leave over a period of 1 year [7, 14, 15].
Until now, the number of studies on work disability and sick leave in early arthritis is confined to the situation after diagnosis, with no data being available on the occurrence of sick leave or permanent work disability in the preceding period. In addition, little is known about the extent of adaptations to the work place or guidance by health professionals in these early stages of the disease.
The aim of this study was therefore to describe the occurrence of sick leave and permanent work disability and its predictors in the very early phases of the disease. Moreover, this study aimed to describe the provision of adaptations to the workplace and professional guidance in patients with chronic arthritis.
Materials and methods
Patients
This work disability study was originally designed as a multicenter study, including the Leiden University Medical Center (LUMC) Leiden, the Haga-Leyenburg Teaching Hospital and Bronovo Hospital, The Hague and the Reinier de Graaf Hospital, Delft, The Netherlands. We aimed to recruit patients in connection with two cohorts, the early arthritis clinic [16] (EAC; LUMC patients only) and the BeSt study [17] (LUMC, Haga-Leyenburg Teaching Hospital, Bronovo Hospital and Reinier de Graaf Gasthuis). BeSt is an abbreviation of treatment strategies (in Dutch: BehandelStrategieën) for RA, with four drug treatment strategies all aimed at achieving low disease activity being compared in patients with early, active RA [17]. The Medical Ethics Committees of all three participating hospitals approved the protocol of this additional work disability study.
Due to logistical problems, the actual recruitment of patients in the period 2000–2003 has only been carried out in connection with the EAC in the LUMC. The EAC is a facility aimed at early diagnosis and treatment of arthritis, including all patients suffering from arthritis in one or more joints, an arthritis symptom duration of less than 2 years, and not being diagnosed with gout, chondrocalcinosis or traumatic arthritis or a serious disease other than a rheumatic condition. General practitioners are requested to refer patients with suspected arthritis as soon as possible; all patients can be seen within 2 weeks after referral [16]. Only patients who had a paid job 12 months before the first visit in connection with the EAC were included.
On average, a definite diagnosis was made within 2 weeks after the first visit in connection with the EAC. At the time the present study was conducted, patients who were diagnosed with RA were promptly treated with either methotrexate or salazopyrine. RA patients fulfilling the inclusion criteria for the BeSt study [RA according to the American College of Rheumatology, formerly American Rheumatism Association, criteria [18], a disease duration of less than 2 years, aged 18 years or older and active disease indicated by ≥6 of 66 swollen joints, ≥6 of 68 tender joints and at least one of the following two criteria: an erythrocyte sedimentation rate (ESR) of ≥28 mm/h or a global health score of ≥20 mm on a 0–100 mm visual analogue scale (VAS), where 0 = best and 100 = worst] were randomized to four treatment groups. These groups included sequential monotherapy (group 1), step-up combination therapy (group 2), initial combination therapy with tapered high-dose prednisone (group 3), or initial combination therapy with infliximab (group 4). For patients failing on their medication, the study protocol prescribed a number of subsequent treatment steps [17].
All patients included in the work disability study gave written informed consent.
Assessment methods
Assessments were done at study entry (at study inclusion of the EAC cohort) and after 12 months of follow-up. Basic sociodemographic and clinical data were obtained from the baseline assessments of the EAC, whereas data on the working situation and mental health were gathered by means of additional questionnaires.
Sociodemographic and clinical characteristics
Sociodemographic variables included age, sex, status of living (living with a partner yes/no) and education level, divided into three categories: primary education (0–8 years), secondary education (9–16 years) and higher vocational education /university (17 years or more).
Disease activity was measured by means of (a) the ESR (mm/h); (b) the Ritchie score, concerning pain in 26 selected joints on a scale of 0–3 (0 = no pain and 3 = worst pain), with the final score ranging from 0–69 [19] and (c) the patient’s global assessment of disease activity, pain and fatigue on three separate VAS. All VAS were 10 cm horizontal lines, with the anchors on the left being no disease activity, no pain and no fatigue, whereas the anchors on the right were the worst imaginable disease activity, severe pain and severe fatigue.
Physical functioning was measured by the Health Assessment Questionnaire (HAQ). The HAQ is a 20-item questionnaire comprising eight domains of activities of daily living, with the final score ranging from 0 (no disability) to 3 (severe disability) [20].
Anxiety and depression were measured by means of a Dutch version of the Hospital Anxiety and Depression Scale (HADS) [21]. The higher the scores on the anxiety or depression scales the more anxious or depressed the subject is (range, 0–21 per scale).
Job characteristics, sick leave, work disability, work adaptations and guidance with working problems
Current occupation was grouped into four categories: (1) predominantly mental demands and absence of physical demands, (2) a combination of physical and mental effort, (3) light physical demands and (4) heavy physical demands [22].
Sick leave was defined as absenteeism with a duration of 2 weeks or more, related to the arthritic complaints and reported to the employer. At the time the study was conducted, in the Dutch social security regulations, the maximum duration of sick leave was 1 year, and sick leave could be either complete (reported sick for all hours that one is engaged) or partial (reported sick for part of the hours that one is engaged). The duration of sick leave in the 12 months before study entry and 1 year thereafter was divided into four categories: (1) 2–4 weeks, (2) 1–3 months, (3) 3–6 months, (4) 6–9 months and (5) 9–12 months.
Permanent work disability was defined as receiving a complete or partial work disability pension related to the rheumatic disease. In the Dutch social security regulations, a partial work disability pension in combination with a part-time job is common. In this situation, sick leave may occur during the hours active at work.
The presence of material or immaterial adaptations at the workplace (including adapted tools, adapted furniture, reduction in tasks or duties, help of colleagues, flexible working hours, opportunities for more breaks, less working hours or reduced work pace) was recorded. Reduced working hours were only considered as an adaptation at work when this adaptation did not pertain to official, incomplete sick leave.
Guidance with working problems was recorded by means of a question concerning the receipt of help in maintaining the job. Health professionals listed for giving help were: the rheumatologist, occupational physician, general practitioner, social worker, physical therapist, occupational therapist or nurse. In addition, it was registered whether patients received support from their employer and/or colleagues. Satisfaction with the received guidance was recorded. In addition, patients were asked by whom they would have liked to be helped more.
Background information on sick leave and work disability
Background information on sick leave and work disability in the Dutch population in the period the study was conducted was derived from the Internet site of the Dutch Central Bureau of Statistics, http://www.cbs.nl.
Statistical analysis
Data entry was performed using Microsoft Office Access 2000. Statistical analyses were executed using SPSS 10.0 for Windows.
Continuous variables were presented as medians with ranges and categorical variables as number and percentages. Comparisons of the median age and the sex distribution of patients included in the study and all other patients of 65 years or younger who entered the EAC cohort in the study period were done by means of the Mann–Whitney U and chi-square test, respectively.
Proportions of patients being on sick leave or receiving a work disability pension at study entry and after 12 months of follow-up were compared using McNemar tests for paired comparisons of categorical data, whereas the average numbers of hours per week patients were working were compared by means of the Wilcoxon signed rank test. These analyses were repeated excluding all patients with a symptom duration >6 months at study entry.
Sociodemographic, disease and job characteristics in the group of patients in whom the working situation deteriorated after 12 months of follow-up (defined as being entitled a partial or full work disability pension for the first time or changing from a partial to a full work disability pension) and in the group of patients in whom the working situation remained stable or improved (defined as receiving no disability pension or the same partial work disability pension at either time point of changing from a partial work disability pension to no disability pension) were compared by Mann–Whitney U and chi-square tests.
Then, a multivariate backward logistic regression analysis on predictors of deterioration of the working status was performed. Only variables that showed a significant difference in the above-mentioned analysis (p values ≤ 0.1) were entered as independent variables in logistic regression analysis.
The numbers of patients receiving adaptations in the workplace or vocational guidance in the past year at study entry and after 12 months of follow up were compared by the McNemar test, whereas the median total numbers of workplace adaptations per patient at both time points were compared by means of the Wilcoxon signed rank test.
Results
In the study period, in total, 418 patients entered the EAC cohort. One hundred and five of these 418 patients (25.1%) were older than 65 years. From the remaining 313 patients, 81 patients (25.9%) were found eligible and willing to participate. The reasons for not being included were not recorded on the individual level. A group of patients was not eligible for the study because they were not having a paid job 12 months before the first visit in connection with the EAC cohort. In addition, a number of patients were eligible but declined participation, and some patients were not invited because of logistic reasons.
Twelve of the initially included patients were excluded within 2 weeks after enrollment because a diagnosis of gout, chondrocalcinosis, sarcoidosis or a non-rheumatic disease was confirmed after additional diagnostic procedures so that they did not meet the inclusion criteria for the EAC.
The 69 patients included in the study were younger (median age, 42.0 years; range 18–64) than the 244 patients in the EAC cohort aged 65 years or less who were not included (median age, 48.0 years; range, 18–65; p = 0.08). Moreover, the proportion of female patients was lower in the 69 included patients (38/69, 55%) than in the 244 patients who were not included (158/244, 65%; p = 0.09).
Sociodemographic and disease characteristics, including physical and mental function, of the 69 included patients are shown in Table 1. In total, 69 patients were recruited from the EAC cohort. Thirty-nine patients (57%) had definite RA, 8 patients (12%) had probable RA, 10 (15%) had psoriatic arthritis, 3 (3%) had spondylarthropathy and 9 (13%) had unclassified arthritis, with the arthritis still being unclassified after 12 months of follow-up. Twelve of the 39 patients with definite RA also participated in the BeSt study; two were randomized to treatment group 1, three to treatment group 2, five to treatment group 3 and two to treatment group 4.
Table 1Sociodemographic and disease characteristics of 69 patients with early arthritisCharacteristicsn (%)Sociodemographic characteristics Age, in years; median (range)42 (18–64) Female patients, n (%)38 (55) Living with partner/family, n (%)57 (83) Educational level, n (%)  High18 (26)  Medium32 (46)  Low19 (28)Disease characteristics Diagnosis, n (%)  RA (definite)39 (57)  RA (probable)8 (12)  Psoriatic arthritis10 (15)  Arthritis unspecified9 (13)  Spondylarthropathy3 (4) Duration of arthritis symptoms, in years; median (range)0.3 (0–2) VAS, 0–10 cm; median (range)  Disease activity patient5.3 (0–10.0)  Pain patient4.7 (1.0–9.8)  Fatigue patient4.8 (0–10.0)ESR, in mm/hr; median (range)20 (2–105)Ritchie index (0–68); median (range)6 (1–32)HAQ (0–3); median (range)0.63 (0–1.88)HADS anxiety; median (range)5 (0–16)HADS depression; median (range)4 (0–13)
In total, 6 patients (9%) had a symptom duration of >1 year, 6 patients (9%) had a symptom duration between 6 months and 1 year and 57 patients (83%) had a symptom duration of <6 months.
Overall, disease activity was moderate, with the median VAS disease activity score being 5.3 (range, 0–10.0) cm, the median ESR 20 (range, 2–105) mm/h and the median Ritchie score being 6 (0–32).
Data on the working status are presented in Table 2. Patients were distributed fairly evenly over the four job categories.
Table 2Working status in 69 patients with early arthritis Study entryFollow-up (12 months)p valueaJob characteristics Occupational category, n (%)  Mental demands18 (26)  Mixed mental/physical demands14 (20)  Light physical demands20 (29) Heavy physical demands17 (25) Hours of work per week; median (range)36 (5–70)34 (4–70)b0.455 Full-time employment, n (%)34 (50)29 (42)0.375Sick leave Current sick leave, n (%)22 (32) Sick leave for the past 12 months, n (%)28 (41)18 (26)0.031 Extent of sick leave, n (%)  Complete sick leave15 (54)12 (67)  Partial sick leave7 (25)3 (17)  Unknown6 (21)3 (17) Duration of sick leave for the past 12 months, n (%)  2–4 weeks5 (18)7 (41)0.754  1–3 months14 (50)2 (12)0.002  3–12 months8 (29)8 (44)1.0  Unknown1 (4)1 (4)1.0Work disability Work disability pension  Complete or partial, n (%)5 (7)13 (19)0.008  Complete, n (%)03 (4)0.250  Partial, n (%)5 (7)10 (15)0.180ap value of Wilcoxon signed rank test or McNemar where appropriate.bOnly patients who did not start to receive a work disability pension or an extension of a work disability pension were considered.
At study entry, 22 patients (32%) were currently on sick leave. In The Netherlands, in the period 2000–2003, the average proportion of persons on sick leave in the working population ranged between 4.8 and 8.4%, depending on the branch (public service or industry) [http://www.cbs.nl]. Twenty-eight patients reported sick leave over the past 12 months, whereas after 12 months follow-up, 18 patients reported sick leave in the preceding period. This difference was mainly accounted for by a decrease in the number of patients reporting sick leave for periods of 1–3 months. In total, 34 patients (49%) reported sick leave in either 12 months before study entry or 12 months thereafter. Of the 27 patients who reported sick leave due to arthritis in the past 12 months with a specific duration at study entry, 5 were partially or fully work disabled after 12 months of follow up, 10 reported long-term sick leave (1–3 months), 4 reported short-term sick leave (2–4 weeks) and 8 reported no sick leave due to arthritis.
Five patients (7%) received a partial disability pension at study entry. In two of these patients, this entitlement was extended to a full work disability pension after 12 months, whereas in the other three patients, the disability pension remained unchanged. In addition, eight patients (12%) started receiving a work disability pension in the 12-month follow-up period (one patient a full disability pension and seven a partial disability pension).
In The Netherlands, the average proportion of employees in the working age range (15–64 years) becoming work disabled over a period of 1 year ranged between 1 and 1.1% in the period the study was conducted (http://www.cbs.nl). None of the patients became unemployed during follow-up.
A repetition of the analyses excluding the six patients with a symptom duration of >1 year showed similar results (data no shown).
Table 3 shows sociodemographic, disease and job characteristics of patients in whom the work status deteriorated (institution or extension of work disability pension) and of patients in whom the working situation improved or remained stable. In the patients in whom the work status deteriorated, the proportions of patients with a diagnosis of definite RA, the occurrence of sick leave in the previous 12 months and a duration of sick leave >4 weeks were significantly higher than in the group of patients in whom the working status remained stable. Moreover, patients in whom the working status deteriorated had a significantly higher VAS disease activity compared to patients in whom the working status remained stable or improved.
Table 3Deterioration of working statusa in 12 months follow up in 69 patients with early arthritis More work disability, n = 10Stable work (dis)ability n = 59p valuebSociodemographic characteristics Age, years; median (range)42 (28–58)44 (18–64)0.484 Female patients, n (%)5 (50)33 (56)0.996 Educational level, n (%)  High5 (50)13 (22)0.141  Medium3 (30)29 (49)0.435  Low2 (20)17 (29)0.846Disease characteristics Diagnosis, n (%)  RA (definite)9 (90)30 (51)0.049  RA (probable)08 (14)0.481  Arthritis unspecified1 (10)8 (14)1.0  Psoriatic arthritis010 (17)0.356  Spondylarthropathy03 (5)1.0 Duration of symptoms, years; median (range)0.4 (0–0.9)0.3 (0–2)0.955 VAS, 0–10 cm; median (range)  Disease activity patient7.5 (4.1–9.8)5.1 (0–10.0)0.038  Pain patient5.7 (2.1–9.8)4.6 (1.0–9.6)0.096  Fatigue patient5.9 (5.0–9.8)4.0 (0–10.0)0.131 Disease activity  ESR, mm/h; median (range)26 (6–105)17 (2–93)0.264  Ritchie index (0–68); median (range)14 (2–17)6 (1–32)0.068 Physical and mental functioning  HAQ (0–3); median (range)0.88 (0–1.75)0.63 (0–1.88)0.174  HADS anxiety; median (range)6.5 (0–13)5 (0–16)0.650  HADS depression; median (range)5 (1–13)3 (0–11)0.195Job characteristics Current occupational category, n (%)  Mental demands3 (30)15 (25)1.0  Mixed mental/physical demands3 (30)11 (19)0.689  Light physical demands1 (10)19 (32)0.292  Heavy physical demands3 (30)14 (24)0.977 Sick leave related to arthritis 12 months before study entry, n (%)9 (90)19 (32)0.002 Duration of sick leave related to arthritis in 12 months before study entry, n (%)  2–4 weeks05 (26)0.767  1–3 months5 (56)9 (47)0.036  3–12 months4 (44)5 (26)0.026aDeterioration of work status defined as starting to receive a work disability pension or altering from a partial to a full work disability pension after 12 months of follow-up.bp value of Mann–Whitney U test or chi-square where appropriate
In the multivariate logistic regression analysis, the occurrence of sick leave in 12 months before study entry was found to be the only significant predictor of deterioration of the working situation (odds ratio, 16.1; 95%CI, 1.8–142.8).
The presence of material and immaterial adaptations at the workplace is shown in Table 4. The number of patients in whom the workplace was adapted increased from 20 (29%) at baseline to 28 patients (42%) after 1 year follow-up. The numbers of patients using adapted tools or furniture, flexible time-management or performing different tasks increased significantly with time. Moreover, the average number of workplace adaptations per patient increased significantly.
Table 4Presence of material and immaterial adaptations at the workplace in 69 patients with early arthritis Study entry, n = 69Follow-up (12 months), n = 66p valueaNo. of patients with adaptations, n (%)20 (29)28 (42)0.134No. of adaptations per personb; median (range)2.5 (1–6)3.0 (1–9)<0.001Types of adaptations Adapted tools/furniture, n (%)1 (3)11 (17)0.002 Different tasks (extra education), n (%)2 (3)12 (18)0.013 Reduction of tasks/duties, n (%)9 (13)18 (27)0.064 Help of colleagues, n (%)9 (13)15 (23)0.210Flexible working hours, n (%)10 (15)7 (11)0.581Flexible time-management, n (%)7 (10)17 (26)0.021Less working hours, n (%)8 (12)12 (18)0.424Reduced work pace, n (%)5 (7)7 (11)0.774ap value of McNemar test or Wilcoxon signed rank test where appropriate.bOnly considering patients with one or more adaptations.
Table 5 shows the guidance of other persons with the aim of the maintenance of the paid job. At study entry, 48 patients (70%) and, in the follow-up period, 36 patients (52%) received professional guidance, with the rheumatologist, general practitioner and physical therapist being the most frequently mentioned health care professionals. The proportions of patients being satisfied with the guidance were the lowest for the occupational physician. Employers and/or colleagues provided help in one third of patients, for the large majority of patients to their satisfaction.
Table 5Usage of and satisfaction with professional guidance and help of employers/colleagues with respect to arthritis-related problems at work in 69 patients with early arthritis Study entryFollow-up (12 months)UsageSatisfactionUsageSatisfactionRheumatologist, n (%)37 (54)35 (95)21 (30)21 (100)General practitioner, n (%)34 (49)29 (85)12 (17) 12 (100)Physiotherapist, n (%)24 (35)20 (83)21 (30)21 (100)Occupational physician, n (%)16 (23)12 (75)19 (28)13 (68)Social worker, n (%)6 (9)6 (100)4 (6)4 (100)Clinical nurse specialist, n (%)6 (9)6 (100)9 (13)8 (89)Occupational therapist, n (%)5 (7)5 (100)9 (13)8 (89)Total48 (70)41 (59)Employer/colleagues, n (%)23 (33)21 (91)21 (30)19 (91)All values expressed as numbers (%)
Thirteen patients (19%) would have liked to have more guidance with problems at work, especially from their rheumatologist, a nurse specialized in rheumatology or from their employer.
Discussion
This study in patients with early chronic arthritis shows that about half of the patients reported arthritis-related sick leave, and 19% became permanently work disabled over a period 1 year before and 1 year after entering an EAC cohort. Partial work disability was more common than full work disability, and sick leave appeared to be its most important predictor. In addition, work place adaptations, professional guidance and help from employer and/or colleagues were common in this population.
With respect to the occurrence of sick leave, our results are in accordance with those of previous follow-up studies in patients with early arthritis, where sick leave rates of 50–60% over a period of 1 year were reported [7, 14, 15]. As in our study, 83% of the patients had a disease duration of less than 6 months; our data on the occurrence of sick leave in the 12 months before study entry suggest that in a considerable proportion of patients, sick leave may occur before they enter an EAC, and a diagnosis of arthritis has been made. In our study, the occurrence of sick leave decreased with time, a finding that cannot be completely interpreted as a positive observation, as it was in part explained by patients on sick leave becoming permanently work disabled (Table 6).
Table 6Working situation after 12 months of follow-up in 27 patients with early arthritis who reported sick leave at study entry (in one patient the duration of sick leave was unknown)Study entryFollow-upDuration of sick leavePartial work disabilityFull work disabilitySick leave 2–4 weeksSick leave 1–3 monthsSick leave 3–12 monthsNo sick leave2–4 weeks (n = 5)0012021–3 months (n = 14)2131343–12 months (n = 8)110042
In our study, 19% of the patients were permanently work disabled after 2 years of follow-up. This figure is in the same range or somewhat more favourable than work disability rates found in previous studies in patients with early RA [7–9]. In fact, our results may even be more favourable, as work disability was partial in the majority of patients, whereas in other studies, only complete job loss was considered. It should be noted, however, that our study concerned various forms of chronic arthritis, and a deterioration of the working situation appeared to be more frequent among patients with a diagnosis of definite RA than in patients with other forms of arthritis. Our results are in concordance with the observation that the incidence of permanent work disability among RA and AS patients appears to be declining over the last decades, most probably related to new developments in pharmacological therapy [23–25]. Indeed, the patients included in our study were promptly treated, with the medical treatment for the patients with RA consisting of either monotherapy with methotrexate or salazopyrine or treatment according to the BeSt protocol [17].
In this study, the duration of symptoms varied among patients, so that not all patients were at equal risk of developing sick leave or work disability. The impact of the symptom duration may however be limited, as one of our analyses showed that symptom duration had no effect on the development of more work disability in the follow-up period of 12 months.
In this study, no control group was used for comparison. Data from the Dutch Central Bureau of Statistics (http://www.cbs.nl) indicate that there is a considerable “background risk” of sick leave and work disability in the general population. However, as in this study, patients were explicitly asked to report sick leave and work disability due to arthritis; it is doubtful whether this background risk needs to be taken into account. It appeared that sick leave and work disability rates among patients with early arthritis were considerably higher than those in the general population; however, for valid comparisons, a separate study with similar registration methods among patients with early arthritis and the general population is needed.
With respect to the determinants of work disability, similar to studies in RA [26], disease severity was, in general higher, and physical and psychological functioning worse among patients in whom the working status deteriorated than in those in whom the working status remained stable. The fact that the association of some measures of disease severity or physical or psychological functioning with changes in the working status did not reach statistical significance may have been due to the relatively small sample size of this study. Moreover, our finding that the occurrence of sick leave in the 12 months before study entry was associated with a deterioration of the working situation is in concordance with the results of another study by our group among another group of patients with various form of chronic arthritis [27]. The results of both studies underscore the importance of the recognition of sick leave as a risk factor for work disability in patients with chronic arthritis. Previously, it has been questioned whether rheumatologists do sufficiently recognize patients’ working problems [28] of which sick leave could be considered as one of its manifestations. Apart from paying attention to the presence of sick leave, the use of instruments that have been specifically developed for the identification of those individuals with chronic arthritis having difficulties at work could be helpful because by using these tools, working problems could be recognized in earlier stages. Examples of those instruments are the RA-Work Instability Scale [29] and the Work Limitations Questionnaire [30]. In addition, Detaille et al. [31] showed which issues are important from the patients’ perspective and recommended a topic list for various health professionals to use during consultations. So far, these instruments are only partly validated, have been mainly used in studies, and are not developed to be applied at the individual level. At present, valid and brief questionnaires specifically designed for clinical practice are not yet available. It is however conceivable that simply asking patients about working problems or sick leave is another sensitive method to identify patients at risk of work disability.
The results of this study show that over a period of 2 years, in 30–40% of the patients, the work situation was adapted, about 30% received help from the employer or colleagues and 60–70% of patients received professional guidance to maintain their job. With respect to the implementation of work place adaptations and help from the employer or colleagues, these figures are in line with those of a recent study in early RA patients [32]. In that study, it was demonstrated that support and help of co-workers, employer help, shortened work days, new skills and a modified work station or equipment were among the work-related factors that significantly ameliorated work ability. With respect to professional guidance, a recent study in Dutch employing RA patients with a short disease duration found that according to the patients, the mobilization of health care professionals would help them tackle their threats to work ability [33].
In our study, patients were generally very appreciative of help concerning their working problems, with the lowest proportions of patients being satisfied obtained for the occupational physician. In The Netherlands, the importance of the role of occupational physician in the maintenance of a paid job in patients with chronic diseases is generally recognized. However, in a previous study, the communication between occupational physicians and rheumatologists was found to be hampered [34], factor that could have contributed to the relatively low satisfaction scores.
A limitation of the present study may be that the total number of patients was relatively low, and various types of chronic arthritis were included, hampering conclusions about specific diagnosis groups. Moreover, in this study, about 25% of all patients in the working age range (<65 years) who entered the EAC participated in the work disability study. This proportion is significantly lower than the percentages of patients with a paid job at symptom onset in similar age groups in early RA cohorts, with frequencies ranging between 50 and 75% [11, 35–37]. As the sociodemographic and clinical characteristics of the non-participants were not systematically gathered, it remains unclear to what extent the population included in this study differs from all patients with early arthritis who have a paid job at symptom onset. Given the probability of selection bias, our results have to be interpreted with care.
In conclusion, this study in patients with early chronic arthritis and a paid job showed that the occurrence of arthritis-related sick leave, before and directly after a diagnosis is made, is substantial. In contrast, full permanent work disability developed in a relatively small minority. Work place adaptations, professional guidance and help from employers or colleagues to maintain a paid job were fairly common. Future research should aim at the detection of patients at risk for the development of sick leave and permanent work disability, as well as the optimization of the process of vocational guidance.

sick leave
work disability
arthritis
occupational health
vocational rehabilitation


Measurement of overall quality of life in nursing homes through self-report: the role of cognitive impairment

Measuring quality of life is a necessity for adequate interventions. This paper concerns the usefulness of six self-report measures for overall quality of life for nursing home residents with various levels of cognitive impairment. It was investigated which proportion of residents from four cognition groups could complete a scale, and internal consistency and construct validity of the scales were studied. Data collection took place in ten Dutch nursing homes (N = 227). The proportion of residents that could complete each scale varied. The Depression List could be administered most often to the cognitively most impaired group (43%; Mini Mental State Examination-scores 0–4). In the three cognition groups with MMSE-score >5, internal consistency of the Depression List, Geriatric Depression Scale and Negative Affect Scale was adequate in all three groups (alpha ≥.68). Intercorrelation was highest for the Philadelphia Geriatric Center Morale Scale, the Depression List, and the Geriatric Depression Scale (rho ≥.65). Nonetheless, self-report scales were not strongly correlated with two observational scales for depression, especially in cognitively severely impaired residents (rho ≤.30). In conclusion, it may not be possible to measure overall quality of life through self-report, and possibly also through observation, in many nursing home residents.
Introduction
Over the past decades, quality of life has become a focal point in scientific research and clinical practice. Although researchers disagree on the domains that make up quality of life, the general consensus is that quality of life measurement should focus on the subjective experience of the individual. This implies that the individual in question is the most valid source of information [1, 2]. However, nursing home residents may not be able to respond to self-report measures or lose this ability during their stay, f.i. due to dementia, which complicates the assessment and monitoring of a resident’s quality of life across time [3–5]. Although self-report is a complex process of introspection and evaluation [6], research has asserted that moderately demented patients still can report on their quality of life, even when they have poor insight into and awareness of their dementia [4, 7–9]. According to Kane et al. [8], 60% of the nursing home population would be able to reliably report on their quality of life. It would, therefore, be helpful to know which scales can be applied to nursing home residents with varying degrees of cognitive impairment.
Most scales measure separate dimensions of quality of life [10]. This has the advantage of a higher responsiveness to change than a measure for overall quality of life (OQOL). However, OQOL is an attractive outcome that can be measured as a single subjective result of weighing unspecified dimensions that are considered to be relevant by the patient. The administration is also less burdening, which is an important factor in a very frail elderly population. Therefore, several researchers (also) use a single overall measure [e.g. 3, 7].
In our approach to quality of life [11, 12], OQOL is represented by subjective (i.e. psychological) well-being. Given this, scales for subjective (psychological) well-being can be used as OQOL scales. Although not always considered as the central outcome, psychological well-being is an important dimension of quality of life in many other approaches to quality of life in the elderly [13–19]. It encompasses both positive and negative affect, and life satisfaction (i.e. morale and contentment) [14, 20, 21], but most often a selection of these concepts is used for measurement. Instruments for psychological well-being that are used in the elderly are, for instance, the Philadelphia Geriatric Center Morale Scale (PGCMS) [22] and the Bradburn Affect Balance Scale [23]. For measuring affect in the elderly, the Positive and Negative Affect Scales [24, 25] and the observational Philadelphia Geriatric Center Positive and Negative Affect Scales [26] have been recommended [21]. Both positive affect and negative affect are important dimensions in quality of life scales [7, 17–19, 27]. Sometimes, however, only negative scales, such as scales for depression, are used in the measurement of quality of life [28]. As the absence of depression does not automatically imply that a resident is happy or content, this poses the question of whether positive and negative scales do, indeed, measure separate constructs and thus, whether or not a negative scale can be used as a single scale for OQOL.
In this paper the aim is to investigate the usefulness of six self-report measurement scales for OQOL, by studying whether they can be administered reliably and validly in a large group of nursing home residents, and whether this is related to cognitive impairment. We hypothesized that if all scales measure OQOL in nursing home patients validly and reliably, they would correlate highly, within all cognition groups. Moreover, the scales should be related to observational scales that measure OQOL.
Methods
Data were collected in ten nursing homes in the Netherlands. The Medical Ethics Committee of the VU University Medical Center had approved the research proposal, and written informed consent was obtained from the participants or their legal representative. Data were collected on a maximum sample of 30 residents over a period of 3 months per nursing home, with an equal distribution of residents with mainly physical handicaps (in so-called ‘somatic’ units) and mainly dementia syndromes (in so-called ‘psychogeriatric’ units).
The principal investigator (DLG, a trained psychologist) administered the self-report OQOL scales and the cognitive test (see later), while the nursing staff carried out the observational assessments. The completeness of the interview data depended on the resident’s cognitive and physical abilities and willingness to answer questions. The scales were administered in random order. The administration of a scale was terminated when a resident proved to be unwilling or unable to respond to the questions that were asked. To assure the validity of cross-sectional comparisons, the self-report and observational scales for each resident were both assessed within the same 4-week period.
Measurement instruments
The scales that were selected to measure OQOL had been used before in published research among nursing home residents or frail elderly populations. A distinction was made between scales that ask about OQOL literally, scales that focus on positive affect, negative affect or life-satisfaction, and scales that indicate clinical depression.
Self-report OQOL scales
A general question on OQOL (GEN-QOLQ) was asked: ‘Overall, how would you rate the quality of your life at the moment?’. This is a modification of the general question on quality of life that is part of Brod and co-workers’ scale for quality of life in people with dementia [7]. To our question ‘at the moment’ was added, because a pilot study showed that, without this explicit time-limit, the residents tended to evaluate the whole of their past life. The response scale, which is presented in the form of a card, consists of the following response categories: 1 = bad, 2 = moderate, 3 = good, 4 = very good, and 5 = excellent.
The Philadelphia Geriatric Center Morale Scale (PGCMS) [22] is a self-report scale that has been developed to assess elderly people in institutions, and has regularly been used as an outcome measure in research on quality of life and well-being in the elderly [e.g. 29, 30]. It consists of 17 dichotomous items measuring life satisfaction, and the scores are summed, with a high score indicating high quality of life. The scale has been found to be reliable, valid and sensitive [30], and internally consistent (KR-20 of .79) [31].
The Positive And Negative Affect Scales (PANAS) [24], were also used. The Positive Affect Scale (PAS) consists of 10 items concerning positive feelings, such as enthusiasm, interest and determination. The Negative Affect Scale (NAS) consists of 10 items concerning ‘negative’ feelings, such as fear, sadness, anxiety and hostility. For this study, the time frame ‘today’ was chosen, and instead of the original 5-category scale, a 2-category response scale was used, because a pilot study showed that very few residents were able to answer the 5-category scale. The administration was visually mediated, following the procedure proposed for the Depression List (see further). Summing the item-scores yielded two separate total scores, ranging from 0 (no positive/negative affects confirmed) to 10 (all positive/negative affects confirmed). The PAS and the NAS were found to be suitable for use in the elderly [21, 25]. Earlier reported internal consistency with the time-frame of ‘today’ yielded a Cronbach’s alpha of .90 for the PAS and .87 for the NAS in the general population [24].
The Depression List (DL) is a Dutch self-report screening instrument for depression, especially suitable for the assessment of (elderly) people with cognitive impairment [32]. It consists of 15 keywords that are presented on cards, one by one, accompanied by a simple question. For instance, a card with ‘down’ printed on it is accompanied by the question ‘do you feel down?’. Sum-scores range from 0 (no depressive complaints) to 30 (many depressive complaints). In psychometric research, the reported internal consistency of the DL was .82 in a group of visitors to a psychogeriatric day-care clinic [32].
The Geriatric Depression Scale (GDS) is a self-report screening instrument for depression in the elderly that is of known reliability and validity, also in long-term care [33, 34]. It consists of 30 dichotomous questions, which are summed into total scores, ranging from 0 (no depressive complaints) to 30 (many depressive complaints).
Other scales
The Mini Mental State Examination (MMSE) is a test for cognition, and has scores ranging from 0 (very severe cognitive impairment) to 30 (no cognitive impairment). It is widely used and has been validated, also in long- term care populations [35, 36].
The GIP-sad behavior [37, 38] is a sub-scale of the Behavior Observation Scale for Geriatric Inpatients (GIP), which is widely used in nursing homes in the Netherlands. The 6-item GIP-sad behavior (GIP-S) measures the behavior of elderly people in intramural care settings that expresses sadness, unhappiness, and anxiety, and is used in the present paper as a scale for OQOL. Sum-scores range from 0 (no sad behavior) to 18 (frequent sad behavior). When first published, the internal consistency (Cronbach’s alpha) of the scale was .84, and the inter-rater reliability (Pearson’s r) was .74 [37]. In a validation study, internal consistency was found to be .86, and the average inter-rater reliability of the items (Cohen’s weighted kappa) was .43 [39].
The MDS Depression Rating Scale (DRS) is an observational scale, based on items from the Minimal Data Set of the Resident Assessment Instrument [40], which can be used to screen for depression [41]. The DRS consists of seven MDS items that are summed. The scores range from 0 (no depressive behavior) to 14 (frequent depressive behavior). The internal consistency (Cronbach’s alpha) when it was developed was .75 in the derivation sample and .71 in the validation sample. Its sensitivity against a psychiatric diagnosis of depression was 91% [41]. In a validity study the internal consistency of the DRS was .68, and its correlations with the Geriatric Depression Scale (GDS) and the Hamilton Depression Rating Scale were .19 and .24 respectively [42].
Analyses
In order to determine whether cognitive status relates to the psychometric properties of the scales, the total group of residents was divided into four MMSE score-groups. An attempt was made to find a division based on known cut-off points that also resulted in equally large groups. The traditional MMSE cut-off point indicating cognitive impairment is 22/23 [43]. Among the reported cut-off points for severe cognitive impairment are 16/17 and 17/18 [36], and known cut-off points for reliable self-report assessment are 9/10 [e.g. 7] and 14/15 [e.g. 44]. The division into cognition groups was carried out as follows: a MMSE score below 5 (very low cognition group); scores from 5 to 12 (low cognition group); scores from 13 to 21 (moderate cognition group); and scores of 22 or higher (‘high’ cognition group).
For each MMSE score-group, the number of residents who could complete each scale was calculated, and compared with the number of residents to whom it was offered. Cronbach’s alphas were calculated to determine the internal consistency of the scales, and were compared across the different cognition groups. Cronbach’s alpha is considered to be fairly good if higher than .70, but should not be higher than .90 [45]. For construct validity, Spearman coefficients for the interrelationships of the self-report OQOL scales and for the relationships of the OQOL scales with the observational scales for OQOL were calculated and compared across the different cognition groups. Significant correlation coefficients are described in the results and interpreted as follows: .21–.40 = fair correlation, .41–.60 = substantial correlation, and .61–.90 = strong correlation.
Results
Sample description
The overall sample consisted of 227 residents. Their average age was 80.5 (SD 9.26; range 52–100), and 78% were female. The average score on the MMSE (N = 200) was 11.8 (SD 9.26); 26.5% had a score below 5, and 18.5% had a score of 22 or higher. The internal consistency of the MMSE in this sample was .89. The curves of the NAS and the DL were slightly positively skewed. The scores on the NAS scales were found to be quite low (Table 1, descriptives total group).
Table 1Descriptives and practical utility of self-report overall quality of life-scalesScales (max. range)Descriptives total groupAvailable MMSE scores% residents who could complete a scale, per MMSE score groupaMean (range)SDNN22–3013–215–120–4PositiveGEN-QOLQ (1–5)2.7 (1–5)1.011911797 (35/36)81 (35/43)80 (43/54)11 (4/36)PGCMS (0–17)10.3 (0–17)4.511211094 (34/36)84 (36/43)72 (39/54)3 (1/36)PAS (0–10)4.4 (0–10)2.610298100 (30/30)97 (31/32)94 (34/36)21 (3/14)NegativeNAS (0–10)1.6 (0–7)1.99894100 (30/30)97 (30/31)94 (31/33)21 (3/14)DL (0–30)8.0 (0–26)5.4143135100 (23/23) 96 (26/27)100 (65/65)43 (21/49)GDS (0–30)10.8 (0–26)7.0106102100 (37/37)91 (39/43)72 (26/36)0 (0/14)aEach fraction presents the number of residents who completed the scale in relation to the total number of residents who were offered the scale. Because the scales were not offered to all 227 participants, the numbers in the denominators do not add up to 227, and differ for each scale
Due to practical considerations and the frailty of the residents, not all scales were administered to all residents. Therefore, the number of completed scales varied.
Proportion of completed scales
Table 1 reports that in the high cognition group (MMSE score of 22 or higher), all six scales could be completed by 94–100% of the residents. In the moderate cognition group (MMSE score between 13 and 21), all scales, except the GEN-QOLQ and the PGCMS, could be completed by 91–97% of the residents. The GEN-QOLQ and the PGCMS could be completed by 84% and 81% in this group, respectively. In the low cognitiongroup (MMSE score 5–12) the DL still could be completed by all residents (100%), the PAS and the NAS by 94%, the GEN-QOLQ by 80%, and the PGCMS and the GDS by 72%. In the very low cognition group (MMSE score below 5) the DL could still be completed by 43% of the residents, but only 21% or less could complete the other scales. The GDS was not completed by any of these residents, and the PGCMS only by 3%.
Only 3 MMSE-groups were used for all further analyses, because in the very low cognition group (N = 53) too few residents could complete the scales.
Internal consistency
The GEN-QOLQ consisted of one question, so internal consistency analysis was not applicable. Table 2 shows that internal consistency of the DL and GDS was satisfactory in all cognition groups; that of the NAS was also acceptable, although this scale was somewhat less consistent in the high cognition group (.68). The PGCMS shows good consistency, except in the moderate cognition group (.53). The PAS is the least reliable scale, as it only reaches an acceptable alpha in the moderate cognition group. Further, the scales showed no linear trends of decreasing internal consistency with increasing cognitive impairment, but there were some variations between cognitive groups.
Table 2Internal consistency of self-report overall quality of life-scales for different cognition groupsTotal groupHigh cognition MMSE ≥22Moderate cognition MMSE <22 & ≥13Low cognition MMSE <13 & ≥5Alphaa (miic) bNcAlpha (miic)NAlpha (miic)NAlpha (miic)NPositivePGCMSd.72 (.17)108.81 (.26)33.53 (.14)35.83 (.22)38PAS.55 (.18)98.47 (.20)29.77 (.25)30.68 (.18)32NegativeNAS.72 (.18)97.68 (.16)29.81 (.31)30.71 (.17)31DL.81 (.22)139.87 (.30)22.72 (.15)26.79 (.21)64GDS.91 (.24)98.90 (.24)34.92 (.29)37.89 (.22)23aAlpha = Cronbach’s alpha; bMiic = mean inter-item correlationcThe Ns of the MMSE-groups do not add up to the N of the total group, because some participants (N = 27 of N = 227) did not have an MMSE-scoredAs it consists of one question, no alpha coefficient could be calculated for GEN-QOLQ
Construct validity
Construct validity, inter-relationship
Table 3 presents the Spearman correlation coefficients for the self-report OQOL scales in the three cognition groups. Each correlation coefficient for two scales pertains to all residents who completed both scales. Therefore, the group-sizes differ for the correlation coefficients. Also, the group sizes may be larger than in Table 2, because scale-scores were also calculated when one of the scale’s items was missing, using mean substitution.
Table 3Spearman correlation coefficients between self-report overall quality of life scales, for the separate MMSE score-groupsGEN-QOLQPGCMSPASNASDLHCaMCLCHCMCLCHCMCLCHCMCLCHCMCLCPositivePGCMS .38*.61**.22N343537PAS.01.30.47**.19.21.34N303031293132NegativeNAS−.06−.32.13−.51**−.69**−.47**−.17−.10.16N302929293030303031DL−.43*−.28−.35*−.75**−.79**−.76**−.36−.27−.25.45.62**.44*N232143232139192134192131GDS−.47**−.60**−.36−.80**−.66**−.65**−.20−.43*−.25.64**.65**.49*.88**.77**.69**N353424343525303023302922232026aHC = High cognition group (MMSE score ≥22); MC = moderate cognition group (MMSE score <22 ≥13); LC = Low cognition group (MMSE score <13 ≥5)* P < .05; ** P < .01
Table 3 shows that the PGCMS, DL and GDS had the strongest intercorrelation. These three scales correlated significantly (P < .01) in all cognition groups. Also the NAS was substantially correlated with the GDS and the DL. The other relationships between the scales varied. As far as the different cognitive groups are concerned, although the correlations between the scales were mostly lowest in the low cognition group, no clear linear trend across cognition groups was visible. The strength of the relationships between the positive PGCMS and the negative NAS, DL and GDS further indicates that these constructs are far from independent, as often has been found.
Construct validity, relationship with GIP-S and DRS
As Table 4 shows, the GIP-S correlated significantly with the GEN-QOLQ, PGCMS, NAS, DL and GDS, but only in the moderate cognition groups. The DRS correlated only significantly to the GEN-QOLQ (only for the high cognition group), PGCMS (for both high and moderate cognition), DL (for moderate cognition) and GDS (for both high and moderate cognition). None of the self-report scales correlated significantly with the observational scales in the low cognition group. Remarkably, there was no correlation between the NAS and the DRS. The correlation coefficients for the GIP-S and the DRS (not shown in Table 4) were .57 in the high cognition group (N = 34); .48 in the moderate cognition group (N = 39); and .42 in the low cognition group (N = 62). In the very low cognition group, in which almost no self-report scales could be administered, the correlation between these two observational scales was .28 (N = 70).
Table 4Relationships of self-report overall quality of life-scales with observational measures for overall quality of life, for the separate MMSE score-groupsPositiveNegativeGEN-QOLQ N = 119PGCMS N = 112PAS N = 102NAS N = 98DL N = 143GDS N = 106HCaMCLCHCMCLCHCMCLCHCMCLCHCMCLCHCMCLCGIP-Srho−.26−.41*−.28−.24−.40*−.06.01.01−.08.36.50**.02.37.42*.14.39.43**.01N = 213N353142343239302833302730232364373525DRSrho−.43*−.25−.09−.36*−.36*−.06.03−.24.11.23.30−.12.12.51*.17.44**.52**.30N = 220N323341323437272932272829222463343725aHC = High cognition group (MMSE score ≥22); MC = Moderate cognition group (MMSE score <22 ≥13); LC = Low cognition group (MMSE score <13 ≥5)* P < 0.05, ** P < 0.01
Discussion
The aim of this paper was to investigate the usefulness of six self-report scales for measuring OQOL in nursing home residents. Hence, it was examined what percentage of residents could complete each scale, how high internal consistency and construct validity of the scales were, and whether these results were associated with level of cognitive impairment. It appeared that, of all the scales considered here, the DL could be administered to the most residents, even to almost half (43%) of the residents with very severe cognitive impairment. The other scales could only be administered to a small minority of this group. However, in the higher cognition groups, all scales (except for the PGCMS in the 5–12 MMSE group) could be administered to 80% to 90% of the residents. This percentage is higher than the 60% that was reported by Kane et al. [8], but refers to shorter scales that only measure OQOL instead of various dimensions of quality of life.
In addition to residents being able to complete a scale, the resulting psychometric properties of the scales are of importance. It appeared that all scales, with the exception of the PAS and, to a lesser extent, the PGCMS, had an acceptable internal consistency. Although the alphas varied across the cognition groups, there was no linear trend of decreasing consistency with increasing cognitive impairment. With regard to validity, the PGCMS, DL and GDS (and the NAS to a lesser extent), were strongly interrelated in all cognition groups, although the correlations in the low cognition group overall were somewhat lower. These four scales also had the strongest relationships with the observational scales for OQOL. However, these relationships were not very strong. Moreover, in the low cognition group the scales were not related to the observational scales. This suggests that level of cognitive impairment has a substantial influence on the validity of self-report OQOL-scales.
The PAS performed worst on all aspects. The low internal consistency in the high cognition group could be explained by one item (‘determined’). When it was omitted, Cronbach’s alpha increased from .52 to .70. Furthermore, its disappointing characteristics may partly be explained by the fact that the PAS measures a somewhat different construct than the other scales. Its low correlation with the NAS, the other PANAS scale, is not unexpected. Positive and negative affect are considered to be largely independent. Considering the item content of the five scales, six affects (items) of the PAS were not included in the PGCMS, the DL or the GDS (i.e. interested, exited, strong, proud, inspired, and determined), whereas only three affects of the NAS were not included (i.e. guilty, hostile, and shameful). The PAS therefore must be considered to measure a different construct than the other scales, which also explains the absence of a correlation with the GIP-sad behavior and the Depression Rating Scale. Although conceptually it may be a good addition to the measurement of OQOL,- as psychological well-being is made up of positive affect, negative affect and life satisfaction -, its poor psychometric performance make it unsuitable in its present form. In contrast, the other PANAS scale, the NAS, had quite good properties, although the mean score on the NAS was low. The rating of affective states ideally involves a consideration of intensity, duration and frequency [21]. However, in cognitively impaired residents it is important to use a self-report scale that is as simple as possible, which may lead to loss of information, and therefore a loss of psychometric quality. For instance, the dichotomized response scale and the time-frame of ‘today’ that was used for the PAS and the NAS may have resulted in less discriminatory and lower scores, but increasing the response categories and the time-frame would threaten its reliability. We therefore suggest further research into the optimal time frame and response categories of the NAS.
Although the DL and the GDS have been developed as screening instruments for depression, they correlated very strongly with the PGCMS, which is a scale for life satisfaction. Studying the item content of the scales, it appeared that the items of these three scales have many similarities. The items of the DL and the GDS contain positive as well as negative affects and also contentment (e.g. ‘satisfied’, ‘happy’ and ‘hopeful’). Remarkably, although the PGCMS is considered to be a positive scale, it has more items that contain negative affects or cognitions (12 out of 17; 71%) than the GDS (20 out of 30; 67%), and especially the DL (7 out of 15; 47%). Therefore, despite the fact that the names of the GDS and the DL suggest a negative scale, they both contain ample positive items. Likewise, the PGCMS cannot be considered to be a solely positive scale. So, with adapted scoring methods, each of these scales could be used as a single measure for OQOL, covering both positive and negative aspects.
As in most proxy studies [2, 46–49], in the present study low correlation was found between self-report and proxy assessment in low cognition groups (in this study: observational scales, rated by the nursing staff). This may suggest that the validity of self-report scales decreases with the level of cognition, but also that the validity of observational scales is lower in the low cognition group. Indeed, the correlation between the two observational scales in this study was lower in the low cognition group. Nevertheless, the relationships between the self-report and observational scales were also not strong in the high cognition group. This suggests that, although they are certainly related, self-report and observational scales may measure a different construct, irrespective of the cognitive functioning of the resident. The fact that both observational scales were negative may have complicated matters even further. Additional research into the relationship of self-report and observational scales for OQOL as well as proxy-report measures (following Edelman et al. and Sloane et al. [46, 48]) in relation to cognitive performance is therefore necessary. This research should use positive observational scales for OQOL, for instance the Philadelphia Geriatric Center Positive and Negative Affect Scales [26]. It should also apply a shorter maximum time-interval for collecting observational data than the four week-period used in this study, since this may have resulted in an underestimation of the association between the self-report scales on the one hand and the observational scales on the other.
For now, the decision as to whether or not a self-report scale can still be administered reliably and validly in a cognitively impaired resident could best be made for individual assessments when administering a quality of life-scale. The research on quality of life measurement in dementia has shown that one can use screening questions, incorporated in the scale, and thus tailored to the specific cognitive demands of the scale, in order to determine whether a resident is cognitively able to answer the questions [7, 50]. Nevertheless, even if a resident appears to be able to understand the questions, it is not certain that her answers are a true reflection of her inner state. Therefore, further research should focus on developing guidelines on when the administration of a scale can be considered as reliable and valid. This research could, for instance, study the possibility of examining test-retest reliability by repeating questions of a scale throughout the assessment. Such test-retest reliability in the assessment of cognitively impaired residents is an important indication that the resident has understood the questions and that she really communicates her true subjective state. In addition, repeating the assessment on another day and calculating a mean score for the two assessments can result in a more stable self-report OQOL score.
In conclusion, measuring overall quality of life reliably and validly through self-report may not be possible in nursing home residents with at least moderate cognitive impairment. The quality of observational assessment of OQOL may also be lower in cognitively impaired residents. Before drawing definite conclusions about the usefulness of self-report scales, it is necessary to study their reproducibility. Nevertheless, in clinical practice, using self-report scales will provide interesting information on the experience of the residents, and is therefore in itself a valuable addition to observational data. The Depression List is a useful scale in this respect, especially for the assessment of nursing home residents with mild to moderate cognitive impairment.

overall quality of life
self-report outcome measures
long term care
age-related memory disorders