How Should We Foster the Professional Integrity of Engineers in Japan ? A Pride-Based Approach I discuss the predicament that engineering-ethics education in Japan now faces and propose a solution to this . The predicament is professional motivation , i.e. , the problem of how to motivate engineering students to maintain their professional integrity . The special professional responsibilities of engineers are often explained either as an implicit social contract between the profession and society -LRB- the `` social-contract '' view -RRB- , or as requirements for membership in the profession -LRB- the `` membership-requirement '' view -RRB- . However , there are empirical data that suggest that such views will not do in Japan , and this is the predicament that confronts us . In this country , the profession of engineering did not exist 10 years ago and is still quite underdeveloped . Engineers in this country do not have privileges , high income , or high social status . Under such conditions , neither the social-contract view nor the membership-requirement view is convincing . As an alternative approach that might work in Japan , I propose a pride-based view . The notion of pride has been analyzed in the virtue-ethics literature , but the full potential of this notion has not been explored . Unlike other kinds of pride , professional pride can directly benefit the general public by motivating engineers to do excellent work even without social rewards , since being proud of themselves is already a reward . My proposal is to foster a particular kind of professional pride associated with the importance of professional services in society , as the motivational basis for professional integrity . There is evidence to suggest that this model works . There is an on-going shift in emphasis in the professional-ethics literature from misconduct to integrity -LSB- 1 , 2 -RSB- . Integrity within this context generally means maintaining high moral standards and doing superior work in a professional capacity , as opposed to merely avoiding misconduct . The profession of engineering is of course no exception . However , even though the importance of professional integrity in society is obvious , the question of motivating students to maintain this integrity is not as straightforward a matter . Engineering-ethics courses are usually expected to do the job . Ethics teachers do not have to spend a great deal of time to explain why professionals should avoid misconduct , as long as students have already acquired everyday morality . Professional integrity and other special responsibilities by professionals , on the other hand , go beyond everyday morality , and need a different basis to convince students . How then can engineers -LRB- and other professionals -RRB- be motivated to provide efficient services to society ? How can we foster professional integrity in engineers ? These issues , which I will henceforth refer to as `` the problem of professional motivation , '' have been discussed by many authors , and many different solutions have been proposed . To clarify the issue , we need to distinguish the motivation to obey the codes of ethics and the justification of such codes . Professional codes of ethics have clear societal virtues , and often essential for the normal functioning of the society . In this sense , the existence of the code is morally desirable and justified . However , there are obligations in codes of ethics that seem to go beyond common sense obligations , such as `` engineers shall be guided in all their relations by the highest standards of honesty and integrity , '' and `` engineers shall at all times strive to serve the public interest , '' both listed as `` professional obligations '' in the Code of Ethics for Engineers by the NSPE -LRB- National Society for the Professional Engineers -RRB- . To use a deontological term , these acts are usually regarded as supererogatory acts for ordinary people , and without special reasons , this should not be different for engineers . Thus , there is a gap between what society needs and what the society can require of engineers as ordinary obligations . The gap can be filled either by justifying the special obligations in a way engineers themselves feel happy with them , or by motivating engineers in other ways to accept these obligations . The problem of professional motivation can be solved and professional integrity will be fostered in either way , but the motivational aspect becomes more crucial when we have to take the second strategy . Since the effectiveness of a solution to the problem is partly dependent on the circumstances engineers are placed in , the most effective solution may differ from one situation to another . One of the main conclusions I intend to draw in this paper is that Japanese ethics teachers face a special predicament that teachers in many other countries -LRB- including the U.S. -RRB- might not . The quandary , if it is real , calls for a different solution to the problem of professional motivation . I will also provide a solution to the problem based on the notion of `` pride '' , which I take to be at least more promising in the current Japanese circumstances than the alternatives . Grounds for Professional Integrity of Engineers Let us first examine some common solutions to the problem of professional motivation . This issue is often discussed within the context of the nature of codes of ethics . Although the scope of professional integrity -LRB- which is my main concern here -RRB- often goes beyond written texts of codes of ethics , for now let us ignore this difference ; for if we fail to persuade students to obey the codes of ethics of relevant professional societies , how can we persuade them to have professional integrity ? Probably the most prevalent view is the so-called `` social contract '' view -LRB- or , more accurately , the `` contract-with-society '' view -RRB- . This view sees an implicit contract between society and the professions and is based on particular notions of what professions are . According to this notion , professions have some characteristic features , such as autonomous professional societies , ethics codes , licenses , and privileges -LSB- 3 , 4 -RSB- . These features are the basis of the contract , where society gives professions prestige , autonomy , educational opportunity , and other support for them to function ; in exchange , professions maintain high moral standards and return to society high-quality services . If students accept this scheme exists , it is rather easy to persuade them to participate in the contract , i.e. , the contract is advantageous for them . This account of professional ethics is widely accepted -LRB- e.g. , Harris et al. -LSB- 5 -RSB- discusses this view -RRB- .1 Another model , which I call the `` membership-requirement '' view , has some strong supporters such as Davis -LSB- 6 , 7 -RSB- . According to this model , engineers have special responsibilities simply because they have decided to take part in the profession , and all members are required to follow special standards -LRB- and the standards in turn are beneficial for the profession as a whole -RRB- . According to this interpretation , professional responsibilities are basically derived from the principle of fairness -LRB- `` do not cheat '' -RRB- , rather than the assumed implicit contract with society .2 Both views have some advantages and disadvantages , but I will not discuss them here . What I would like to ask is a rather different question : can these views justify special obligations of engineers and motivate engineers to have a sense of professional integrity , especially in Japan ? There are some reasons for believing that the situation is apparently different in this eastern country , and this is what I will try to demonstrate in the following . Japanese Peculiarities The peculiarities of the Japanese situation I have in mind involve the environment Japanese engineers are placed in . Both the `` social-contract '' view and the `` membership-requirement '' view presuppose the existence of the profession of engineering as a matter of fact -LRB- rather than as an ideal -RRB- , and the former view further presupposes the existence of an implicit contract between the profession and society -LRB- again , as a matter of fact -RRB- . However , neither of these conditions is actually met in Japan . Underdevelopment in Profession of Engineering First , until quite recently , engineers in Japan had never characterized themselves as professionals in a relevant sense . As a matter of fact , the very notion of profession was foreign to them . Engineering societies in Japan are not professional organizations in the proper sense , but academic societies ; their main purpose is to organize academic meetings , and they rarely play a role in defending the interests of engineers . In fact , the lack of the notion of profession in engineering is frequently mentioned in the engineering-ethics literature in Japan as the main obstacle to the development of professional engineering ethics -LSB- 8 , 9 -RSB- . Since they were -LRB- and still largely are -RRB- academic societies , most of the societies never had an ethics code -LRB- until recently -RRB- . There was no accreditation system for engineering education , and professional engineer -LRB- PE -RRB- licenses did not exist . This was the situation in Japan 10 years ago . Now , there is an on-going change toward the professionalization of engineering in Japan . Engineering societies have started to establish codes of ethics . An organization for accreditation , the Japan Accreditation Board of Engineering Education -LRB- JABEE -RRB- , was established in 1999 and some engineering departments have already been accredited . To meet the accreditation criteria , engineering schools started to offer engineering-ethics courses , one of which I am teaching myself . The government amended the licenses for Consulting Engineers into those for Professional Engineers in 2000 , and introduced some new features , such as the Continual Professional Development -LRB- CPD -RRB- requirement , to the licenses .3 Since licensing and accreditation have been used by various professions to attain privilege and autonomy , these changes seem to be steps toward the professionalization of engineering in Japan . However , the flip side of these positive developments is that those who are actively involved in these changes are still in the minority . Such changes are mainly caused by external factors irrelevant to most engineers . An effort was made in 1997 to make Japanese engineers internationally competitive .4 To achieve this purpose , the authorities had to establish an accreditation system compatible with ABET -LRB- Accreditation Board for Engineering and Technology -RRB- of the United States and other Washington Accord countries , but one conspicuously missing element in the Japanese engineering-education system was ethics education . They studied engineering-ethics education systems in other countries -LRB- especially in the U.S. -RRB- , and noticed that such systems are firmly based on the notion of profession . They thus started the series of changes in professionalization . International competitiveness and the establishment of an accreditation system are not problems of their own making for most individual engineers in Japan . Even though codes of ethics as written texts have been established , what gives life to such texts is the members of the societies who consciously observe them , and in this sense the codes do not yet have real lives . It will take a long time for Japanese engineers to become aware of the importance of codes of ethics and regard engineering as a profession in the proper sense . Even after the recent amendments , there are still almost no privileges associated with PE licenses , and most engineers can still conduct their business without any license whatsoever .5 Many engineering schools -LRB- including most of the best engineering programs in the country -RRB- are not yet accredited , but their graduates do not seem to have any trouble finding jobs . Thus , the change toward professional privilege and autonomy is still nominal . In summary , no autonomous professional groups existed for engineering students to take part in 10 years ago , and even now these are quite underdeveloped . Anyway , there are no advantages gained by becoming a member . Given these considerations , the `` membership-requirement '' view is hopeless as a clue to motivating engineering students , and therefore does not seem to solve the problem of professional motivation , at least for the time being , in Japan . This also means that an important part of the supposed community deal in the `` social-contract '' view is also missing . Do we have to wait until the profession is firmly established before teaching professional ethics ? Lack of Implicit Social Contract in Japan Even if engineering is not a profession in the proper sense , if there is an implicit social contract between engineers and society as a matter of fact , the `` social-contract '' view will do . Privilege and autonomy are not the only things given to engineers by society . For example , if their social status is sufficiently high , that can be an implicit term in the social contract . However , there is evidence that suggests that the social status of engineers is not as high as they might wish . In the following , let us look at a couple of suggestive surveys in this regard . There was an interesting survey conducted in 1998 by Matsushige et al. -LSB- 10 -RSB- that showed that Japanese engineers do not earn that much -LRB- at least not enough to make the `` social-contract '' view persuasive -RRB- .6 Osaka University is one of seven major national -LRB- former Imperial -RRB- universities in Japan . They were classified by the degree they earned , and four of the categories were : Bachelor of Engineering , Master of Engineering , Bachelor of Social Science , and Bachelor of Humanities . They were further divided into age groups and the average salaries were calculated for all categories . The results for male graduates are summarized in Table 1.7 Table 1Average salary in 1998 of Osaka University male graduates -LRB- reproduced from -LSB- 10 -RSB- , p. 5 -RRB- 22 -- 29 years old30 -- 39 years old40 -- 49 years old50 -- 60 years oldBachelor of Social Sciences ¥ 4,626,000 -LRB- $ 38,600 -RRB- ¥ 9,133,000 -LRB- $ 76,100 -RRB- ¥ 13,067,000 -LRB- $ 108,800 -RRB- ¥ 15,682,000 -LRB- $ 130,700 -RRB- Bachelor of Engineering ¥ 5,088,000 -LRB- $ 42,400 -RRB- ¥ 7,036,000 -LRB- $ 58,600 -RRB- ¥ 10,988,000 -LRB- $ 91,500 -RRB- ¥ 14,566,000 -LRB- $ 121,400 -RRB- Master of Engineering and higher ¥ 4,370,000 -LRB- $ 36,400 -RRB- ¥ 7,392,000 -LRB- $ 61,600 -RRB- ¥ 11,457,000 -LRB- $ 95,500 -RRB- ¥ 14,004,000 -LRB- $ 116,700 -RRB- Bachelor of Humanities ¥ 4,844,000 -LRB- $ 40,300 -RRB- ¥ 7,365,000 -LRB- $ 61,300 -RRB- ¥ 11,792,000 -LRB- $ 98,300 -RRB- ¥ 12,329,000 -LRB- $ 102,700 -RRB- Dollar amounts are an approximation with $ 1 = ¥ 120 at the current rate There are several things to note here . First , social-science-major graduates are consistently better paid than other majors after they reach their thirties . Up to their forties , humanities majors are paid as much as engineering majors . In their fifties , the average salary of humanities graduates becomes lower than that of engineering graduates . However , according to the analysis done by Matsushige et al. , this difference is not caused by recognizing the status of engineers , but by the tendency for engineers to stay at a single company , while humanities majors leave their original company at around this age .8 Given that engineering majors spend longer in doing course work than social science or humanities majors , there is no point in majoring in engineering if one is interested in earning a good salary . There seems to be no recognition that the status of engineers is higher than that of the others in terms of salary . Of course , the data should be compared with similar data from other countries to analyze it objectively , especially that from the U.S. , but so far I have found no comparable surveys .9 We also should note that as Osaka University is by no means an average university , it is possible that these results will not apply to less prestigious universities . Still this is a very suggestive case study . Even if they do not earn much , it is still possible for engineers to be highly regarded in society . However , another social survey in 1995 -LRB- part of a large survey called Social Stratification and Mobility -RRB- suggested that the social status of engineers is not as high as that of other professions -LSB- 11 -RSB- . Four-thousand ordinary people were asked to rank the social status of 56 occupations , and `` automobile design engineers '' was one of them . This occupation was ranked 14th in the 56 occupations -LRB- Table 2 -RRB- . This may not sound that serious , but other professions , such as doctors , judges , architects and accountants ranked much higher than engineers .10 Table 2Occupational Prestige Scores in 1995 `` Stratification and Social Mobility '' Survey in Japan -LRB- reconstructed from -LSB- 11 -RSB- ; the original result has 56 jobs listed -RRB- Doctor 90.1 -LRB- highest -RRB- Nurse 59.7 CEO of large company 87.3 Police officer 57.9 Lawyer 86.9 Municipal officer 56.8 University professor 84.3 Carpenter 53.1 Congressman 74.9 Restaurant cook 51.6 Architect 71.9 Mechanical assembly worker 51.1 Accountant 70.8 Farmer 45.6 Manager of small company 68.9 Insurance sales person 44.2 Automotive design engineer 66.3 Waitress 38.0 School teacher 63.5 Coal miner 36.6 -LRB- lowest -RRB- A similar poll on occupational prestige is conducted in the U.S. every year by Harris Interactive , and engineers in the 2006 poll had higher prestige than professions such as architects , athletes , lawyers , and accountants , even though firefighters , doctors , and nurses had yet higher prestige -LRB- Table 3 -RRB- .11 This high status for engineers has been fairly stable since this poll began in 1977 . Since the questions in the Japanese and American surveys were different , we can not simply compare the two surveys ; but the difference in the trend is remarkable . Table 3The percentage of people who think that the occupation has `` very great prestige '' . Based on survey conducted by Harris Interactive in 2006Firefighter 63 % Athlete 23 % Doctor 58 % Lawyer 21 % Nurse 55 % Entertainer 18 % Scientist 54 % Accountant 17 % Teacher 52 % Banker 17 % Military officer 51 % Journalist 16 % Police officer 43 % Union Leader 12 % Priest/Minister/Clergyman 40 % Actor 12 % Farmer 36 % Business executive 11 % Engineer 34 % Stockbroker 11 % Member of Congress 28 % Real estate agent/broker 6 % Architect 27 % To sum up , most elements of the `` social-contract '' view , i.e. , the engineering profession , codes of ethics , privilege , high income , and prestige , are missing or underdeveloped in Japan . How can there be a social contract under such circumstances ? Thus , this view does not seem to solve the problem of professional motivation in Japan either . Pride-based Approach Outline of Pride-based Approach What I have argued so far is that both the `` social-contract '' view and the `` membership-requirement '' view lack a factual basis in Japanese society . This also means that they are hard to use in engineering-ethics education in Japan . What then are the alternatives available in Japan , especially in teaching professional integrity ? An appeal to common morality may do for many items in codes of ethics . For example , the avoidance of harm to the public can be derived from the harm principle . However , as I pointed out at the beginning of this paper , there are other obligations in codes of ethics that seem to go beyond common sense obligations , and the `` social contract '' and `` membership requirement '' are invoked because common morality do not seem sufficient for engineers . The model of ethics education I will outline in the following is based on the notion of pride . The basic idea is simple . One can have pride as a professional even where the profession as an autonomous body is absent . Such pride will help engineers to adhere to high moral standards , even without being rewarded by society . We can expect to promote moral behavior in engineers -LRB- including engineering students -RRB- by developing their professional pride in engineering-ethics education .12 In terms of justification , this model does not try to justify special obligations of engineers ; rather , the model tries to motivate students to accept supererogatory acts as their own obligations . Concept of Professional Pride To explicate this approach , I need to say more about the very notion of pride in general , and professional pride in particular . First , let us examine the notion of pride itself . Pride is a positive feeling toward oneself and things associated with oneself . Let us call these things the `` object '' of pride , following David Hume , whose analysis of this notion is one of the earliest -LSB- 13 -RSB- .13 Groundless pride might be psychologically possible , but pride usually involves some beneficial properties about the object of pride as grounds -LRB- again , following Hume , let us call this the `` cause '' of pride -RRB- . Since Hume , several moral philosophers have focused on this notion , especially in terms of virtue ethics . Some authors have considered pride as a vice -LRB- i.e. , as contrary to the virtue of modesty -RRB- , and others as a virtue -LSB- 14 , 15 -RSB- .14 When virtue ethicists -LRB- including Hume -RRB- talk about pride as a virtue , their reason for the positive evaluation of pride is based on its effect on oneself ; pride is , according to these authors , a virtue because it promotes self-esteem and personal happiness . Of course , pride as a virtue is being considered within the present context . Just like other virtues in virtue ethics , such as courage , a misplaced pride and an excessive pride can be a vice rather than a virtue . Thus , just like a courage shown in a robbery does not count as a virtue , a pride as a robber does not count as a virtue . It is hard to make precise the distinction between misplaced and appropriate prides . This is a problem with the virtue ethics approach in general .15 This may be true of pride in general , but professional pride in particular can have a different other-regarding moral function . Professionals in various professions seem to have pride in their trade . What kind of pride is it ? The object of professional pride may be the professional him or her as a professional , the profession as a collective body , or other professionals in the same profession . The cause of professional pride may be excellence in professional skill , the importance of the service they have done to society , or possibly high moral standards . To be justly proud of these things , professionals have to maintain these qualities in their professional work . This will of course benefit society . Here , we can see the possibility for professional pride to be an other-regarding virtue . The beautiful part of this scheme is that society does not necessarily have to reward good work ; the feeling of pride itself functions as a reward . Of course , such feelings will be reinforced by the recognition of excellence by others .16 When we think of professional integrity , professional pride has another virtue . Since pride is a positive feeling , it can motivate professionals to overachieve , i.e. , to do things that are not required or expected -LRB- they overachieve simply because it feels good -RRB- . This is in strong contrast to negative feelings , such as fear of punishment and feelings of guilt , which do not motivate professionals to achieve more than required . Given these limitations of negative feelings , some sort of positive feelings will be necessary as motivation in the transition from minimal professional responsibility to professional integrity , and professional pride is a good candidate for this . Let us now examine whether this will solve the problem of professional motivation in Japan . Given that the concept of the engineering profession as an autonomous body is absent in Japan , the profession itself may not be suitable as an object of pride . However , we do have individual engineers , and of course they can be the objects of their own professional pride . Some possible causes of professional pride -LRB- such as the high moral standards of the profession as a whole -RRB- are missing in Japan , but other important ones , such as their excellence as engineers and the importance of services they give to society , do exist . Rewards from society , which will reinforce this pride , can hardly be expected in Japan , but , as I said , pride is self-rewarding . Thus , the pride-based approach seems -LRB- at least -RRB- a possible solution to the problem . Prospects for Pride-based Education The pride-based approach is yet to be developed . There are many problems to be solved and many questions to be answered before the model outlined here can be implemented in engineering-ethics education . First , does real-world professional pride have desirable characteristics ? It is possible for professionals to feel pride because of something other than their skills or services . For example , the sole cause of their pride may be their high income and prestige . If this is the case , the pride-based model does not work where the `` social-contract '' view does not . This is a psychological question that calls for an empirical investigation . Second , can professional pride become strong enough to support professional integrity ? The `` social-contract '' view seems to provide a fairly strong motivational basis for professional ethics , where it works . Can pride be as strong as such motivation ? This is another psychological issue , but harder to answer , given the speculative nature of the question . Third , is it possible to teach pride in the first place ? Is pride not something we acquire by ourselves , rather than something taught in school ? This is a pedagogical inquiry , which calls for classroom experiments . There is an interesting survey that may illuminate some of these issues . Okamoto et al. surveyed firefighters in Japan on their occupational self-esteem -LSB- 17 -RSB- . The results revealed that there were two independent factors in the sense of occupational self-esteem firefighters had -LRB- they conducted factor analysis on the replies to their questionnaire -RRB- . The first was the esteem based on the service they gave , which Okamoto et al. called `` self-esteem from occupational tasks '' , and the second was the esteem stemming from the very nature of the occupation -LRB- e.g. , intense training and dangerous work -RRB- , which they called `` self-esteem from occupational capacity '' -LRB- pp. 37 -- 38 -RRB- . They further found that the self-esteem from occupational tasks was negatively correlated with delinquency , while the correlation between the self-esteem from occupational capacity and delinquency was not statistically significant -LRB- pp. 51 -- 53 -RRB- . Since there are many differences between firefighting -LRB- which is not even a profession in the strictest sense -RRB- and engineering , we need to use a great deal of caution in applying the results of Okamoto et al. 's study . However , their results constitute positive empirical evidence that occupational self-esteem -LRB- which is often associated with , or even equated with , pride -RRB- can be caused by the importance of the service being provided , and that that kind of self-esteem helps the person to maintain high moral standards . If this is true , there are practical implications for engineering-ethics education under the pride-based approach . The best way to foster the desirable kind of professional pride in engineering students in the classroom is to stress the importance of the service provided by engineers . Professional pride acquired in this way will be conducive to professional integrity . Placing emphasis on professional capacity -LRB- e.g. , higher education and skill -RRB- may also foster professional pride , but we can not expect this kind of pride to be related to professional integrity . Even though I limited my discussion to Japan , it is plausible that many aspects of my discussion apply to other countries . Professionals play similar roles in other countries , and it is only natural that they have the similar motivational structure . Thus , in countries where the social contract model does not work , the pride-based approach may be a solution . In countries where we can assume the social contract between the society and engineering profession , we may not need the pride-based model , but still it may help in motivating engineering students . To summarize , the notion of pride is a practical one , and engineering-ethics education based on this concept seems promising . Although we have to accumulate more empirical data and undertake more conceptual analyses before drawing conclusions on the effectiveness of such approaches , this approach is worth trying in a country where the profession of engineering and its implicit social contract with society are almost absent .

O O O O O B I O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

engineering ethics
professional integrity
japan
pride


Quantification

of

the

level

descriptors

for

the

standard

three-level

system

and

a

five-level

version

according

to

two

methods

Objectives

Our

aim

was

to

compare

the

quantitative

position

of

the

level

descriptors

of

the

standard

three-level

system

-LRB-

-RRB-

and

a

newly

developed

experimental

five-level

version

-LRB-

-RRB-

using

a

direct

and

a

vignette-based

indirect

method

Introduction

The

is

a

widely

used

instrument

to

describe

and

value

generic

health

-LRB-

status

-RRB-

in

terms

of

five

dimensions

mobility

self-care

usual

activities

pain/discomfort

and

anxiety/depression

Each

dimension

comprises

three

levels

indicating

no

problems

some

or

moderate

problems

and

extreme

problems

resulting

in

a

total

of

-LRB-

-RRB-

unique

health

states

-LSB-

-RSB-

The

condensed

format

of

the

has

undoubtedly

contributed

to

its

global

dissemination

as

it

is

easy

to

include

in

existing

surveys

by

questionnaire

designers

easy

to

fill

out

by

respondents

and

easy

to

report

by

analysts

However

compared

with

other

generic

preference

based

instruments

such

as

the

Health

Utilities

Index

Mark

and

Mark

-LRB-

and

-RRB-

and

the

Short

Form

-LRB-

-RRB-

which

define

respectively

and

unique

health

states

the

is

lacking

descriptive

richness

-LSB-

-RSB-

Although

the

descriptive

system

has

demonstrated

strong

psychometric

properties

in

general

its

restricted

ability

to

discriminate

-LRB-

clinically

relevant

-RRB-

small

to

moderate

differences

in

health

status

between

individuals

or

within

individuals

over

time

is

recognized

-LSB-

-RSB-

Moreover

several

studies

have

reported

on

the

ceiling

effect

of

the

in

the

general

population

as

well

as

in

patient

populations

-LSB-

-RSB-

A

straightforward

way

of

improving

the

discriminatory

potential

of

the

descriptive

system

is

to

increase

the

number

of

response

options

In

most

health-status

classification

systems

the

response

options

are

ordered

in

terms

of

severity

along

a

hypothetical

measurement

continuum

Since

the

exact

position

of

the

response

options

defines

the

discriminatory

abilities

of

the

descriptive

system

-LSB-

-RSB-

it

is

important

to

know

where

on

the

measurement

continuum

the

level

descriptors

are

quantitatively

positioned

Previous

research

in

which

a

five-level

-LRB-

-RRB-

version

of

was

compared

with

the

standard

three-level

-LRB-

-RRB-

demonstrated

increased

discriminatory

power

increased

reliability

and

satisfactory

validity

-LSB-

-RSB-

This

paper

presents

a

head-to-head

comparison

of

the

quantitative

positioning

of

the

level

descriptors

of

the

standard

descriptive

system

versus

a

newly

developed

experimental

system

which

covers

unique

health

states

-LRB-

-RRB-

Two

independent

methods

were

used

The

first

method

directly

compared

the

nonextreme

level

descriptors

-LRB-

for

the

level

two

midcategory

for

the

level

two

level

three

and

level

four

categories

-RRB-

for

each

dimension

separately

on

a

visual

analogue

scale

-LRB-

VAS

-RRB-

The

second

indirect

method

required

respondents

to

score

complete

health

scenarios

-LRB-

vignettes

-RRB-

on

dimension-specific

VAS

scales

and

subsequently

to

classify

the

same

vignettes

on

the

two

instruments

-LRB-

and

-RRB-

Methods

Instruments

Three

instruments

were

used

in

this

study

the

standard

version

an

adapted

Dutch

version

developed

in

-LSB-

-RSB-

and

a

set

of

five

dimension-specific

VAS

scales

The

version

of

the

used

in

this

study

was

an

experimental

version

since

at

the

time

of

this

study

no

official

five-level

version

had

been

advocated

by

the

EuroQol

Group

We

chose

to

test

a

five-level

system

even

though

we

also

could

have

chosen

four

or

six

levels

An

increase

in

the

number

of

levels

is

always

an

increase

of

discriminatory

potential

at

the

cost

of

a

more

complex

descriptive

system

-LRB-

which

might

compromise

the

robustness

of

the

value

function

-RRB-

Five

levels

appears

to

be

an

optimal

number

of

response

options

concerning

reliability

-LSB-

-RSB-

Furthermore

Preston

et

al.

-LRB-

-RRB-

investigated

feasibility

for

different

rating

formats

-LRB-

ranging

from

to

and

a

point

scale

-RRB-

and

found

that

feasibility

peaked

at

five

levels

-LSB-

-RSB-

We

chose

to

add

two

in-between

levels

to

the

existing

descriptive

system

-LRB-

between

levels

and

and

levels

and

-RRB-

because

we

considered

this

the

most

obvious

option

in

regard

to

the

objective

of

refining

the

instrument

In

any

preference-based

instrument

level

descriptors

are

practically

required

for

valuation

research

in

which

generic

profiles

are

to

be

valued

A

small

focus

group

was

assigned

to

determine

the

wording

of

the

level

descriptors

The

level

descriptors

presented

here

were

translated

from

Dutch

The

one

three

and

five-level

descriptors

in

were

the

same

as

the

one

two

and

three

level

descriptors

in

the

standard

The

grading

terms

that

were

used

for

the

intermediate

levels

two

and

four

in

the

were

a

little

for

level

-LRB-

-RRB-

in

Anxiety/Depression

and

mild

problems

for

the

remaining

dimensions

and

severe

for

level

-LRB-

-RRB-

in

Pain/Discomfort

very

for

Anxiety/Depression

and

many

problems

for

the

remaining

dimensions

One

further

alteration

was

made

to

both

the

and

systems

the

most

severe

response

category

in

Mobility

was

changed

from

confined

to

bed

to

unable

to

walk

about

so

it

would

be

analogous

to

the

extreme

response

categories

of

the

other

dimensions

Table

displays

the

exact

wording

of

the

descriptors

in

the

and

systems

respectively

Table

quantification

of

three

and

five

level

-LRB-

-RRB-

No

problems

in

walking

abouta

Some

problems

in

walking

Unable

to

walk

abouta

Self-care

No

problems

with

self-carea

Some

problems

washing

or

dressing

Unable

to

wash

or

dress

selfa

Usual

activities

No

problems

with

performing

usual

activitiesa

Some

problems

with

performing

usual

Unable

to

perform

usual

activitiesa

Pain/Discomfort

No

pain

or

discomforta

Moderate

pain

or

Extreme

pain

or

Worst

imaginable

pain

or

discomforta

Anxiety/Depression

Not

anxious

or

depresseda

Moderately

anxious

or

Extremely

anxious

or

Worst

imaginable

anxiety

or

depressiona

No

problems

in

walking

abouta

Mild

problems

in

walking

Some

problems

in

walking

Many

problems

in

walking

Unable

to

walk

abouta

Self-care

No

problems

with

self-carea

Mild

problems

washing

or

dressing

Some

problems

washing

or

dressing

Many

problems

washing

or

dressing

Unable

to

wash

or

dress

selfa

Usual

activities

No

problems

with

performing

usual

activitiesa

Mild

problems

with

performing

usual

Some

problems

with

performing

usual

Many

problems

with

performing

usual

Unable

to

perform

usual

activitiesa

Pain/Discomfort

No

pain

or

discomforta

Mild

pain

or

Moderate

pain

or

Severe

pain

or

Extreme

pain

or

Worst

imaginable

pain

or

discomforta

Anxiety/Depression

Not

anxious

or

depresseda

A

little

anxious

or

Moderately

anxious

or

Very

anxious

or

Extremely

anxious

or

Worst

imaginable

anxiety

or

depressiona

CI

confidence

intervalaLevel

descriptor

used

as

anchor

in

visual

analog

scale

To

obtain

quantitative

values

for

each

level

descriptor

of

and

the

VAS

was

used

We

used

five

VAS

scales

one

for

each

dimension

Each

VAS

consisted

of

a

horizontal

hashmarked

line

without

corresponding

numbers

with

the

extreme-level

descriptors

belonging

to

that

dimension

as

anchors

Respondents

were

asked

to

indicate

their

score

on

the

VAS

by

marking

the

line

For

the

most

severe

category

of

Pain/Discomfort

and

Anxiety/Depression

the

original

descriptor

was

labeled

extreme

Because

the

study

was

part

of

a

larger

process

of

choosing

the

definite

level

descriptors

for

the

official

five-level

version

of

the

we

decided

to

use

the

entire

continuum

of

disability

-LRB-

extreme

included

-RRB-

and

used

worst

imaginable

as

upper

VAS

anchor

for

these

two

dimensions

This

is

analogous

to

the

other

three

dimensions

which

ranged

from

no

problems

to

unable

to

Study

design

Data

collection

took

place

in

the

form

of

one

of

two

panel

sessions

and

a

follow-up

postal

survey

weeks

later

A

convenience

sample

of

laypeople

from

an

existing

general

population

panel

-LRB-

N

-RRB-

participated

All

participants

were

familiar

with

the

vignette

presentation

form

used

in

the

indirect

method

All

participants

completed

both

the

direct

and

the

indirect

quantification

task

For

the

direct

method

all

answers

were

obtained

during

the

panel

sessions

and

all

answers

as

part

of

the

postal

survey

to

avoid

memory

effects

For

the

indirect

method

participants

scored

ten

health

states

in

the

panel

sessions

-LRB-

acute

pharyngitis

exacerbation

of

eczema

hip

fracture

cerebrovascular

accident/stroke

with

moderate

impairments

moderate

gastritis

low

spinal

cord

lesion

mild

depression

back

and

neck

pain

severe

dementia

and

acute

multiple

injury

-RRB-

and

the

remaining

five

in

the

survey

-LRB-

otitis

externa

severe

stable

brain

injury

irritable

bowel

syndrome

acute

large

burn

and

posttraumatic

stress

disorder

-RRB-

because

we

expected

that

more

than

ten

health

states

within

one

session

could

lead

to

concentration

problems

The

two

sets

of

health

states

were

balanced

according

to

severity

and

duration

Following

this

design

the

indirect

method

provided

responses

for

each

respondent

diseases

dimensions

response

scales

Direct

quantification

of

level

descriptors

In

the

direct

method

respondents

were

asked

to

project

the

and

the

descriptors

on

the

VAS

scales

for

each

dimension

separately

As

the

extreme

levels

were

used

as

anchors

of

the

VAS

for

only

the

midcategory

-LRB-

-RRB-

level

descriptor

needed

to

be

scored

except

for

Pain/Discomfort

and

Anxiety/Depression

which

needed

additional

scoring

of

-LRB-

extreme

-RRB-

Similarly

the

midcategories

and

descriptors

were

scored

for

each

dimension

except

for

Pain/Discomfort

and

Anxiety/Depression

which

included

the

scoring

of

Indirect

quantification

of

level

descriptors

As

an

alternative

to

the

direct

method

we

developed

an

indirect

method

that

we

believe

lies

closer

to

the

actual

use

of

the

instrument

as

it

uses

a

-LRB-

hypothetical

-RRB-

health

state

as

a

calibrator

or

medium

to

derive

a

VAS

score

In

contrast

to

the

direct

method

the

object

of

measurement

in

the

indirect

method

is

not

a

or

descriptor

but

a

complete

health

scenario

-LRB-

vignette

-RRB-

Each

vignette

was

scored

with

the

and

descriptors

and

on

a

VAS

one

for

each

separate

dimension

independently

Consequently

an

indirect

head-to-head

comparison

of

and

scores

could

be

made

calibrated

via

the

common

VAS

score

Figure

shows

one

of

the

vignettes

Each

vignette

was

designed

to

present

a

disease

as

close

to

clinical

reality

as

possible

therefore

also

including

information

on

disease

duration

All

diseases

were

presented

on

a

standardized

sheet

-LRB-

vignette

-RRB-

that

contained

-LRB-

-RRB-

a

disease

label

with

a

naturalistic

description

of

the

disease

-LRB-

-RRB-

the

course

of

the

disease

over

a

period

using

a

calendar

-LRB-

the

grey

scales

represent

the

duration

of

the

disease

-RRB-

-LRB-

-RRB-

the

location

of

the

disease

with

if

relevant

a

visual

representation

and

-LRB-

-RRB-

the

dimensions

of

which

the

levels

were

left

unspecified

as

the

respondents

were

invited

to

select

the

appropriate

level

-LRB-

according

to

his

or

her

own

view

-RRB-

for

each

dimension

Respondents

were

asked

to

read

each

vignette

carefully

and

to

select

the

level

of

each

dimension

of

the

descriptive

system

that

best

described

the

presented

health

state

in

their

view

using

three

response

scales

the

standard

response

scale

the

new

scale

and

the

VAS

scale

-LRB-

similar

to

the

VAS

used

in

the

direct

method

-RRB-

Fig.

vignette

with

empty

descriptive

system

The

and

response

scales

were

presented

on

the

left

and

the

right

side

of

one

page

-LRB-

per

dimension

-RRB-

respectively

The

respondents

were

first

invited

to

score

the

descriptors

for

all

dimensions

and

all

vignettes

while

covering

the

right

side

of

the

page

that

showed

the

descriptors

Next

they

were

instructed

to

return

to

the

first

vignette

asked

to

cover

the

left

side

with

the

scores

and

provide

the

response

for

all

vignettes

Pilot

testing

revealed

that

when

respondents

scored

first

there

was

a

tendency

to

avoid

the

in-between

levels

and

of

and

for

this

reason

all

respondents

were

asked

to

score

first

Adequate

instruction

was

critical

stressing

that

and

were

two

independent

ways

of

scoring

-LRB-

in

the

postal

survey

these

instructions

were

repeated

in

writing

-RRB-

Subsequently

VAS

scores

were

obtained

on

a

separate

form

without

respondents

having

access

to

the

and

scores

The

demanding

task

of

first

providing

classifications

on

all

five

dimensions

of

all

vignettes

minimized

possible

memory

effects

when

the

participants

were

instructed

to

return

to

the

first

vignette

to

score

the

classifications

while

covering

the

responses

Analysis

Results

of

the

direct

and

indirect

methods

are

presented

with

conventional

descriptive

statistics

Results

of

the

indirect

method

were

derived

by

grouping

pairs

and

pairs

for

each

respondent

per

vignette

and

subsequently

by

calculating

level

means

over

all

vignettes

and

all

respondents

combined

For

each

respondent

scorings

were

removed

for

the

combined

and

VAS

scores

if

at

least

one

of

the

or

VAS

scores

was

missing

equalizing

the

number

of

VAS

observations

between

and

Characteristics

For

both

the

direct

and

indirect

methods

the

extension

of

was

investigated

in

terms

of

three

characteristics

First

equidistance

addresses

the

degree

to

which

and

level

descriptors

are

distributed

evenly

over

the

VAS

continuum

either

without

or

with

transformation

Equidistance

is

determined

for

each

dimension

and

each

instrument

-LRB-

and

-RRB-

separately

Untransformed

equidistance

implies

that

level

descriptors

are

distributed

according

to

VAS

ratings

of

for

and

for

There

is

evidence

that

the

precision

of

the

VAS

might

be

illusory

as

respondents

mentally

divide

the

VAS

continuum

in

a

smaller

number

of

segments

which

is

nine

or

ten

at

maximum

-LSB-

-RSB-

Therefore

we

defined

a

deviation

of

VAS

points

as

the

maximum

acceptable

deviation

-LRB-

which

makes

a

segment

of

VAS

points

as

the

deviation

can

be

either

way

-RRB-

Furthermore

a

deviation

of

VAS

points

has

been

used

before

-LSB-

-RSB-

If

untransformed

equidistance

is

rejected

equidistance

using

power

-LSB-

y

-LRB-

ax

-RRB-

b

-RSB-

transformation

is

considered

A

power

relation

of

e.g.

y

-LRB-

x

-RRB-

for

would

result

in

a

VAS

rating

distribution

of

Note

that

transformation

is

only

possible

for

as

there

is

only

one

observation

apart

from

the

anchors

Part

of

the

evaluation

of

equidistance

is

analysis

of

the

position

of

the

extreme

levels

according

to

the

indirect

method

are

the

VAS

ratings

for

the

extreme

level

descriptors

close

to

the

supposed

anchor

values

for

the

indirect

method

Ideally

and

scores

would

equal

and

and

scores

would

equal

except

for

Pain/Discomfort

and

Anxiety/Depression

in

which

the

and

extreme

level

descriptors

were

not

identical

to

the

VAS

anchors

Second

isoformity

is

the

degree

to

which

the

positions

of

and

level

descriptors

-LRB-

and

also

versus

for

Pain/Discomfort

and

Anxiety/Depression

-RRB-

are

similar

Isoformity

directly

compares

the

and

descriptive

systems

for

each

separate

dimension

between

instruments

For

the

indirect

method

all

level

means

including

and

can

be

compared

with

Analysis

of

isoformity

is

based

on

paired

response

means

for

each

dimension

separately

For

the

direct

method

isoformity

was

tested

with

a

paired

t

test

between

the

and

scorings

For

the

indirect

method

a

deviation

of

VAS

points

was

defined

as

the

maximum

acceptable

deviation

Finally

consistency

between

dimensions

is

the

degree

to

which

the

positions

of

the

same

level

descriptors

differ

across

dimensions

Consistency

between

dimensions

was

tested

for

each

instrument

-LRB-

-RRB-

separately

The

first

three

dimensions

-LRB-

Mobility

Self-Care

and

Usual

Activities

-RRB-

were

distinguished

from

the

last

two

-LRB-

Pain/Discomfort

and

Anxiety/Depression

-RRB-

as

these

in

Dutch

share

identical

level

descriptors

e.g.

some

problems

for

Mobility

Self-Care

and

Usual

Activities

For

the

direct

method

analysis

of

variance

-LRB-

ANOVA

-RRB-

was

used

for

each

identical

level

descriptor

for

the

first

three

dimensions

combined

-LRB-

one

comparison

for

and

three

for

-RRB-

and

Pain/Discomfort

and

Anxiety/Depression

combined

-LRB-

two

comparisons

for

and

four

for

-RRB-

resulting

in

a

total

of

ten

comparisons

For

the

indirect

method

consistency

is

tested

with

a

generalizability

study

-LRB-

G-study

-RRB-

In

a

G-study

one

is

able

to

separate

multiple

sources

of

error

variance

-LSB-

-RSB-

Generalizability

coefficients

-LRB-

G-coefficients

-RRB-

can

be

constructed

as

functions

of

the

estimated

variance

components

expressing

consistency

on

a

scale

with

expressing

perfect

consistency

-LSB-

-RSB-

We

used

a

variance

components

analysis

based

on

the

restricted

maximum

likelihood

method

and

identified

four

possible

sources

of

variance

label

vignette

dimension

and

respondent

Four

separate

G-studies

were

conducted

one

on

the

first

three

dimensions

and

one

on

the

remaining

two

dimensions

for

each

instrument

-LRB-

-RRB-

separately

A

G-coefficient

expressing

consistency

between

dimensions

was

calculated

on

the

basis

of

these

variance

components

-LRB-

Appendix

A

-RRB-

We

regarded

transformed

or

untransformed

equidistance

to

be

a

desirable

characteristic

for

the

new

system

as

opposed

to

no

systematic

relation

between

the

quantitative

position

of

the

level

descriptors

at

all

Consistency

between

identical-level

descriptors

across

dimensions

was

also

regarded

as

a

desirable

property

because

this

expresses

that

respondents

have

a

consistent

conceptualization

of

the

grading

terms

used

over

different

dimensions

of

health

When

consistency

is

achieved

this

does

not

imply

that

utility

values

would

also

be

expected

to

be

consistent

over

dimensions

because

utility

values

are

an

expression

of

an

entire

profile

whereas

we

investigated

VAS

scores

within

each

dimension

separately

Furthermore

a

choice-based

method

presumably

leads

to

different

results

than

the

dimension-specific

VAS

scales

we

used

We

investigated

isoformity

to

see

whether

the

new

system

was

a

refinement

or

a

new

system

and

whether

isoformity

was

achieved

or

not

does

not

tell

us

anything

about

the

system

in

itself

Results

The

mean

age

of

the

participants

was

years

with

being

men

Of

the

respondents

who

attended

in

the

panel

sessions

returned

the

survey

Three

respondents

-LRB-

-RRB-

were

of

Turkish

nationality

two

-LRB-

-RRB-

were

of

Moroccan

nationality

and

the

remaining

-LRB-

-RRB-

were

of

Dutch

origin

In

the

Pain/Discomfort

and

Anxiety/Depression

dimensions

respondents

often

failed

to

score

the

extreme-level

descriptor

when

using

the

direct

method

-LRB-

and

for

respectively

and

and

for

respectively

-RRB-

For

these

respondents

the

remaining

scorings

were

deleted

for

that

dimension

because

of

possible

context

effects

-LRB-

i.e.

spreading

out

the

VAS

scores

of

the

remaining

descriptors

over

the

VAS

scale

-RRB-

For

the

direct

method

missing

responses

for

ranged

from

-LRB-

Usual

Activities

-RRB-

to

-LRB-

Pain/Discomfort

-RRB-

and

for

from

-LRB-

Usual

Activities

-RRB-

to

-LRB-

Pain/Discomfort

-RRB-

For

the

indirect

method

missing

responses

ranged

from

-LRB-

Usual

Activities

-RRB-

to

-LRB-

Pain/Discomfort

-RRB-

for

the

three

response

scales

-LRB-

and

VAS

-RRB-

combined

Characteristics

direct

method

Results

for

the

direct

method

are

shown

in

Table

and

Fig.

Untransformed

equidistance

was

rejected

for

all

level

descriptors

except

in

Mobility

-LRB-

-RRB-

although

Self-Care

and

Usual

Activities

were

only

VAS

point

away

for

Mobility

Regardless

of

dimension

level

descriptors

were

positioned

systematically

lower

than

the

expected

value

for

equidistance

for

-LRB-

VAS

points

lower

-RRB-

-LRB-

points

lower

-RRB-

and

level

-LRB-

points

lower

-RRB-

whereas

was

sometimes

higher

-LRB-

points

-RRB-

and

sometimes

lower

-LRB-

points

-RRB-

Transformed

equidistance

-LRB-

power

function

-RRB-

provided

an

excellent

fit

for

all

dimensions

of

-LRB-

-RRB-

Fig.

quantification

of

the

three

and

five-level

-LRB-

-RRB-

descriptors

Visual

analog

scale

-LRB-

VAS

-RRB-

means

by

dimension

Isoformity

could

not

be

established

except

for

the

middle-level

descriptors

-LRB-

vs.

-RRB-

for

Pain/Discomfort

and

Anxiety/Depression

-LRB-

Table

-RRB-

Relatively

large

gaps

appeared

between

and

for

Mobility

-LRB-

-RRB-

Self-Care

-LRB-

-RRB-

and

Usual

Activities

-LRB-

-RRB-

with

showing

systematically

higher

values

Although

there

was

a

statistically

significant

difference

between

the

extreme

level

descriptors

-LRB-

vs.

-RRB-

for

Anxiety/Depression

the

absolute

difference

was

VAS

points

Table

of

identical

three-and

five-level

-LRB-

-RRB-

descriptors

for

the

direct

quantification

methodDimensionComparisonMean

differenceP

Usual

Consistency

between

dimensions

gives

supportive

results

for

both

and

as

of

the

ten

comparisons

-LRB-

ANOVA

-RRB-

showed

significant

differences

-LRB-

see

Fig.

-RRB-

Generally

VAS

means

are

similar

among

the

first

three

dimensions

as

well

as

among

Pain/Discomfort

and

Anxiety/Depression

Characteristics

indirect

method

Results

of

the

indirect

method

are

shown

in

Table

and

Fig.

Untransformed

equidistance

of

was

rejected

for

all

dimensions

-LRB-

systematically

VAS

points

too

low

-RRB-

as

well

as

for

-LRB-

systematically

points

lower

-RRB-

and

-LRB-

systematically

points

lower

-RRB-

Untransformed

equidistance

was

achieved

only

for

the

level

for

all

dimensions

-LRB-

systematically

points

lower

-RRB-

with

VAS

scores

ranging

from

-LRB-

Mobility

and

Usual

Activities

-RRB-

to

-LRB-

Anxiety/Depression

-RRB-

Transformed

equidistance

-LRB-

power

function

-RRB-

provided

an

excellent

fit

for

all

dimensions

of

-LRB-

-RRB-

Table

quantification

of

three

and

five-level

-LRB-

-RRB-

Mobility

No

problems

in

walking

Some

problems

in

walking

Unable

to

walk

No

problems

with

Some

problems

washing

or

dressing

Unable

to

wash

or

dress

Usual

activities

No

problems

with

performing

usual

Some

problems

with

performing

usual

Unable

to

perform

usual

Pain/Discomfort

No

pain

or

Moderate

pain

or

Extreme

pain

or

Anxiety/Depression

Not

anxious

or

Moderately

anxious

or

Extremely

anxious

or

LMobility

No

problems

in

walking

Mild

problems

in

walking

Some

problems

in

walking

Many

problems

in

walking

Unable

to

walk

No

problems

with

Mild

problems

washing

or

dressing

Some

problems

washing

or

dressing

Many

problems

washing

or

dressing

Unable

to

wash

or

dress

Usual

activities

No

problems

with

performing

usual

Mild

problems

with

performing

usual

Some

problems

with

performing

usual

Many

problems

with

performing

usual

Unable

to

perform

usual

Pain/Discomfort

No

pain

or

Mild

pain

or

Moderate

pain

or

Severe

pain

or

Extreme

pain

or

Anxiety/Depression

Not

anxious

or

A

little

anxious

or

Moderately

anxious

or

Very

anxious

or

Extremely

anxious

or

CI

confidence

intervalFig

quantification

of

the

three

and

five-level

-LRB-

-RRB-

descriptors

Visual

analog

scale

-LRB-

VAS

-RRB-

means

by

dimension

VAS

results

for

the

extreme-level

descriptors

show

that

the

lower

extreme

is

close

to

except

for

Pain/Discomfort

-LRB-

-RRB-

VAS

results

for

the

upper

extreme

values

are

systematically

higher

for

than

for

-LRB-

range

of

difference

-RRB-

Noticeable

are

large

deviations

in

Self-Care

-LRB-

-RRB-

and

Usual

Activities

-LRB-

-RRB-

Isoformity

was

accepted

for

versus

for

all

dimensions

and

for

vs.

for

all

dimensions

except

Mobility

-LRB-

showing

a

gap

of

points

-RRB-

Isoformity

was

rejected

for

the

upper

extreme

comparison

versus

for

all

dimensions

Consistency

between

dimensions

gave

supportive

results

for

both

and

Table

shows

the

G-study

results

Most

variance

is

attributed

to

the

label

component

whereas

less

than

of

variance

is

attributed

to

the

components

including

dimension

which

is

reflected

in

high

G-coefficients

for

all

comparisons

Consistency

for

is

somewhat

higher

-LRB-

-RRB-

than

for

-LRB-

-RRB-

Table

between

dimensions

for

the

indirect

quantification

method

Variance

components

estimates

-LRB-

percentages

-RRB-

and

generalizability

coefficients

-LRB-

G-coefficients

-RRB-

for

comparable

dimensions

of

three

and

five-level

-LRB-

-RRB-

Label

Label

Label

Label

Label

Label

Vignette

Vignette

Vignette

Vignette

Dimension

Dimension

respondent0.60Residual12.09Residual12.34G-coefficient0.86

Pain/Discomfort

Label

Label

Label

Label

Label

Label

Vignette

Vignette

Vignette

Vignette

Dimension

Dimension

respondent0.45Residual16.78Residual13.42G-coefficient0.81

Discussion

In

this

study

we

compared

the

quantitative

position

of

the

level

descriptors

of

the

standard

and

a

new

five-level

version

using

two

independent

methods

The

study

showed

that

the

extension

of

the

to

a

five-level

version

by

inserting

two

extra

levels

leaving

the

existing

descriptors

unaltered

is

not

a

simple

refinement

but

a

redesign

The

inserted

levels

pushed

the

extreme

levels

closer

to

the

anchors

which

indicates

that

makes

better

use

of

the

measurement

continuum

contributing

to

superior

descriptive

power

of

the

version

In

both

the

and

versions

the

position

of

the

or

descriptors

reassuringly

was

independent

of

dimension

Equidistance

was

not

achieved

for

both

systems

in

most

cases

showing

values

lower

than

the

equidistant

values

Both

methods

revealed

a

large

gap

between

the

and

levels

regardless

of

dimension

This

could

be

caused

by

the

wording

of

-LSB-

some

and

moderate

-LRB-

ly

-RRB-

-RSB-

being

interpreted

as

fairly

mild

In

Pain/Discomfort

respondents

tended

to

avoid

the

lower

anchor

of

the

scale

indicating

some

pain

or

discomfort

on

VAS

while

scoring

no

problems

on

and

This

indicates

that

respondents

preferred

a

more

refined

response

scale

for

scoring

pain

or

discomfort

maybe

a

scale

with

even

more

than

five

response

options

-LRB-

as

is

the

case

of

e.g.

the

or

-RRB-

Also

noticeable

were

the

gaps

observed

for

the

upper

extreme

in

Self-Care

for

which

we

can

not

provide

an

explanation

Isoformity

between

and

showed

mixed

results

The

vs.

descriptors

showed

isoformity

-LRB-

indirect

method

only

-RRB-

as

expected

as

these

both

indicated

the

upper

ceiling

-LRB-

no

problems

-RRB-

Isoformity

was

also

established

for

the

middle

level

descriptors

of

Pain/Discomfort

and

Anxiety/Depression

for

both

methods

This

could

be

due

to

the

wording

of

the

middle

level

descriptors

as

the

descriptor

some

problems

represented

a

wider

range

and

hence

more

potential

variation

than

moderate

-LRB-

ly

-RRB-

as

used

in

Pain/Discomfort

and

Anxiety/Depression

Assuming

that

the

descriptor

some

problems

was

a

well-considered

choice

in

the

development

of

the

original

system

in

order

to

cover

the

entire

range

between

the

two

extremes

it

is

questionable

whether

that

descriptor

is

still

suitable

in

a

version

Direct

quantification

is

a

well-known

method

of

estimating

the

magnitude

of

level

descriptors

or

response

labels

-LSB-

-RSB-

This

approach

however

ignores

the

fact

that

the

VAS

values

expressed

for

the

level

descriptors

did

not

necessarily

reflect

the

self-report

use

of

such

descriptors

-LRB-

and

the

use

in

subsequent

valuation

studies

-RRB-

in

a

similar

way

because

the

valuation

of

an

abstract

level

descriptor

might

lead

to

different

results

than

self-reported

health

The

indirect

method

is

novel

to

our

knowledge

this

is

the

first

time

a

quantification

of

level

descriptors

is

estimated

with

this

method

The

indirect

method

has

several

advantages

First

we

believe

it

is

a

better

representation

of

the

hypothesized

measurement

continuum

of

as

the

medium

of

the

vignette

-LRB-

disease

-RRB-

was

used

to

calibrate

and

descriptors

on

a

VAS

scale

Second

it

is

closer

to

the

general

use

of

the

instrument

as

a

self-report

health

status

assessment

measure

and

is

therefore

likely

to

be

more

valid

Classifying

a

vignette

can

be

regarded

similarly

to

a

health

status

classification

by

proxy

assessment

Other

advantages

of

the

indirect

method

are

analytical

values

can

be

calculated

for

all

level

descriptors

including

the

anchors

and

it

is

possible

to

investigate

explained

variance

for

various

components

-LRB-

G-study

-RRB-

Furthermore

the

indirect

method

proved

to

be

much

more

feasible

than

the

direct

method

considering

the

lower

number

of

missing

responses

Disadvantages

are

that

no

direct

comparison

-LRB-

e.g.

paired

t

test

-RRB-

between

and

is

possible

as

there

is

only

one

VAS

value

for

each

response

pair

and

that

the

indirect

method

is

more

time

consuming

A

potential

weakness

of

the

study

procedure

is

that

and

were

presented

on

one

sheet

and

panelists

were

asked

to

score

dimensions

first

while

covering

and

vice

versa

We

can

not

be

sure

that

respondents

actually

complied

to

the

blinding

procedure

in

the

follow-up

measurement

Also

there

might

have

been

an

order

effect

as

always

preceded

The

instrument

presented

here

obviously

improves

the

discriminatory

potential

of

the

descriptive

system

as

the

level

descriptors

generally

capture

a

larger

part

of

the

measurement

continuum

and

broaden

the

measurement

space

Furthermore

showed

slightly

better

consistency

between

levels

In

a

previous

study

we

demonstrated

increased

discriminatory

power

of

the

same

version

of

as

well

as

superior

reliability

-LRB-

interobserver

and

test

retest

-RRB-

and

face

validity

when

compared

with

the

standard

-LSB-

-RSB-

Awaiting

a

valuation

study

for

an

official

version

of

a

set

of

preference

weights

was

developed

for

this

version

of

using

item

response

theory

-LRB-

IRT

-RRB-

methodology

-LSB-

-RSB-

An

officially

sanctioned

five-level

descriptive

system

will

become

available

within

a

short

period

-LSB-

-RSB-

and

is

expected

to

be

in

use

alongside

the

standard

three-level

The

experimental

five-level

version

presented

here

is

likely

to

demonstrate

a

less

severe

ceiling

effect

Assuming

that

milder

states

are

more

common

in

the

general

population

we

expect

increased

benefit

in

the

detection

of

mild

problems

and

in

measuring

and

monitoring

general

population

health

although

the

extra

level

is

expected

to

also

lead

to

better

differentiation

and

detection

of

more

severe

health

states

The

methodology

presented

here

can

be

of

use

in

the

development

of

generic

or

disease-specific

health

status

measures

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O

eq-5d
health-related quality of life
psychometrics