社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

原著

Relationships between Cognitive Appraisals of Elderly Welfare Policy,
Program, and Project for the Elderly at Home
在宅高齢者の高齢者福祉政策・施策・事業に対する認知的評価間の関係
Ryosuke DEI1) 2), Masafumi KIRINO3), Kazuo NAKAJIMA4), Takashi MURAKOSO3)
出井涼介 1) 2)、桐野匡史 3)、中嶋和夫 4)、村社卓 3)
1) Graduate School of Health and Welfare Science, Okayama Prefectural University
2) Research Institute of Community Care Management
3) Faculty of Health and Welfare Science, Okayama Prefectural University
4) Professor Emeritus, Okayama Prefectural University
1) 岡山県立大学大学院保健福祉学研究科
2) 地域ケア経営マネジメント研究所
3) 岡山県立大学保健福祉学部
4) 岡山県立大学名誉教授

Abstract
The purpose of this study was to reveal the relationships between cognitive appraisals of elderly welfare policy,
program, and project for the elderly at home. We conducted a self-administrated questionnaire survey of 600 elderly
subjects at home in Y-city, X-prefecture. The survey items contained basic attributes and cognitive appraisals of elderly
welfare policies, programs, and projects. We analyzed the causal model using structural equation modeling with datafrom 244 elderly subjects. Consequently, the causal model fits the data (CFI = 0.983, RMSEA = 0.067), and there wasa statistically significant relevance between the components of the causal model. The results suggest that a policy
evaluation based on the logic model can reflect the evaluation of the policy by citizens.
抄録
本研究は、在宅高齢者の高齢者福祉政策・施策・事業に対する認知的評価間の関係を明らかにすることを目的とした。調査は、A 県 B 市の在宅高齢者 600 人を対象に質問紙調査を実施した。調査内容は対象者の基本属性（性別・年齢）、高齢者福祉政策・施策・事業に対する認知的評価で構成した。統計解析には欠損値を有さない 244 人分のデータを使用し、構造方程式モデリングを用いて因果関係モデルを検討した。その結果、因果関係モデルはデータに適合し（CFI = 0.983、RMSEA = 0.067）、因果関係の要素間に統計学的に有意な関連性が認められた。以上の結果は、ロジックモデルに基づく政策評価に対し、市民の政策に対する評価を反映させられる可能性を示唆している。
Keywords: policy evaluation, logic model, structural equation modeling
キーワード：政策評価，ロジックモデル，構造方程式モデリング－－社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

Ⅰ . Introduction

modeling (SEM) (hereinafter, logic model evaluation).

The outcomes of elderly welfare policies, such as

Furthermore, although the above guidelines show thethe allocation of appropriate resources and services,

importance of evaluation based on the viewpoints ofand the improvement of the quality-of-life for the

citizens and users (hereinafter, citizens), few studies

elderly are necessary for welfare issues occurring

are found that use the citizen’s evaluation index

in super-aged-society. An appropriate evaluation is(hereinafter, the citizen index) for policy, program,essential to measure the policy outcomes. In 2001,

and project. We believe an examination of policy

the Japanese government enacted the“Government

evaluation by index, with a scientific basis that

Policy Evaluation Act”aimed at promoting effective,

sufficiently reflects citizens’ evaluations, is an urgent

efficient operations and ensuring accountability torequirement.

the people by objective policy evaluations. Following

This study aims to reveal the relationships between

this, the Cabinet approved the“Basic Guidelines forcognitive appraisals of elderly welfare policy, program,

Implementing Policy Evaluation” and formulated

and project for the elderly at home, with the intention“Guidelines for Implementation of Policy Evaluation”2)of obtaining basic required information that contributes

1)to the systematization of policy evaluation.

in 2005. These documents emphasize policy
management based on the plan-do-check-act cycle
and the quantitative grasp of policy effects. However,Ⅱ . Methods

they do not show concrete and scientific evaluation

1. Subjects

methods. In Japan, various institutions conduct policy

The subjects are 600 elderly people at homeevaluation; however, their analytical method is not

in Y-city, X-prefecture. We conducted a self-consistent and their scientific basis is insufficient.

administrated questionnaire survey. First, we

In summarizing studies on policy evaluation,

explained the purpose and ethical considerations of3)

the study in writing and in an information session

which systematically evaluates policy effects and theto the 20 chiefs of the social welfare council branch4)-7)

causes of problems as a basis for policy evaluation.

in Y-city and we received their consent for the

Program evaluation includes a theory evaluation stage

survey. Second, we sent an explanation in writingthat examines the policy’s logical structure.3) The aimto the elderly at home who were the subjects of the

8)of the theory evaluation is to develop a“logic model,”survey and distributed the questionnaire. We judged

which includes a chain of causal relationships thatparticipation in the survey by whether there waslead to the achievement of policy objectives baseda reply from the questionnaire. We conducted theon the theory of program evaluation (program

survey for two months beginning in June 2015.

researchers emphasize “program evaluation,”

theory).

9)-11)

The causal relationship hypothesized

2. Survey Questionnaires

as a logic model requires appropriate evidence.3) In

The questionnaire contained items on basic

previous studies, some reports develop a logic model

attributes (sex, age) and a cognitive appraisal of thealong the framework and discuss how to develop

elderly welfare policy.

it. 12)-16) However, few studies empirically examine

In the survey, considering the policy structure, we

the developed logic model. Moreover, there are few

measured the cognitive appraisal of the welfare policy

studies in which the components of the logic model—on three scales: 1) policy, 2) program, and 3) project.

in the broad sense of policy dimension, including

For these scales, we prepared original questionnaire(narrow sense) policy, program, and project—are

items with the aim of scale development that canfaithfully scaled and the suitability of the assumed

be used as a citizen index in policy evaluations of

causal chain itself is examined by structural equation

municipalities nationwide. Regarding the design of－－社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

items, we compiled the matters described in the

project is satisfying their needs.

“Act on Social Welfare for the Elderly,”“LongTerm Care Insurance Act” and the municipal

3. Statistical analysis

action plan of the whole country and used it as a

We constructed the causal model reflecting thereference. Furthermore, we also referred to previous

policy structure of elderly welfare policy in thestudies.

17)-19)Regarding the structure of scales, we

impact theory of logic model based on program

designed the survey so each scale has a conceptual

theory and conducted a statistical analysis. Impact

unidimensionality. We compiled the elements of thetheory, which refers to a part of the logic model,

policy’s basic philosophy listed in abstract sentencesshows a causal relationship that outcome (effect ofin the municipal action plan. Based on that, we

the improvement to beneficiaries) is caused by policy

constructed the cognitive appraisal scale for policy

output.9) As an analysis model, we constructed thewith four items measuring the achievement level ofindirect effect model in which the long-term outcome

policy’s basic philosophy in the citizens’ view. We set(dependent variable) was the achievement level of thea four-point scale for the answers (0: not at all, 1: notpolicy’s basic philosophy, the medium-term outcome

very much, 2: a little, and 3: very much). This means(primary factor) was the progress of program, andthat the higher the score, the more the citizen feels

the short-term outcome (secondary factor) wasthat the policy’s basic philosophy has been achieved.

the satisfaction of needs by project. We adopted

We prepared five items measuring the progress ofstructural equation modeling as a statistical analysis

program in the citizens’ view from the viewpoint ofmethod and examined the suitability of the analysis

whether the enhancement of basic programs listedmodel for the data and the relevance between

in the nationwide action plan supports the elderly at

variables. When analyzing the causal model, wehome and constructed the cognitive appraisal scale

introduced sex and age as control variables for thefor program with these items. We set a four-point

purpose of separating the effects of other variables

scale for the answers (0: not at all, 1: a little, 2: quite,from the relationship between each outcome.

and 3: sufficient). This means that the higher the

Prior to the analysis described above, we examinedscore, the more the citizen feels that the program

the construct validity of the scales in the study byis progressing. We prepared 28 items measuringconfirmatory factor analysis. We assumed that thesatisfaction of needs by project in the citizens’ view

cognitive appraisal scale for policy is a one-factor

from the viewpoint of whether the output of eachmodel, the cognitive appraisal scale for program is a

program placed under the project described above

one-factor model, and the cognitive appraisal scale forfunctions effectively to solve the citizen’s needs andproject is a second-order factor model.

constructed the cognitive appraisal scale for projectIn the above analysis, we used weighted least

with these items. We assumed the structure ofsquares with mean and variance adjustment

this scale to be five factors. Distribution of factors(WLSMV).20) As for model suitability, we assessed

and items were as follows: three items for healththe fit of the factor structure model and the causal

promotion support, four items for social participation

model above to the data with comparative fit index

support, 11 items for regional life support, three items(CFI)21) and root mean square error of approximation

for preventive long-term care service, and seven(RMSEA).22) In general, there are no crucial problems

items for enhancement of long-term care service. We

caused by the adoption of the model, on the condition

set a four-point scale for the answers (0: not at all, 1:that CFI is over 0.90 and RMSEA is under 0.10. We

a little, 2: quite, and 3: sufficient). This means that theutilized Mplus 7.3 for statistical analysis.

higher the score, the more the citizen feels that theIn the study, we received responses from 340 out－－社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

of 600 elderly subjects (response rate: 56.7 % ). For

Table 2 shows the answer distributions of the
progress of program in the citizens’ view.

statistical analysis, of the 340 responses, we utilized

The suitability to the data from the one-factor model

244 questionnaires that did not have missing values.

related to the cognitive appraisal scale for program was
CFI > 0.999, RMSEA < 0.001 (Figure 2). We assumed

Ⅲ . Resultscorrelation between yB1 and yB2’s error variables.

1. Distribution of Basic Attributes

Table 3 shows the answer distributions of

The distribution of the sex of the participants

satisfaction of needs by project in the citizens’view.

was 104 male (42.6% ) and 140 female (57.4% ), andTable 2 Distribution of Answers to Items Related to theProgress of Program

the distribution of age was 73.0 mean (SD: 6.08)
with a range of 65–96.

2. Answer Distribution and Validity of Scale,WHPV

Table 1 shows the answer distributions ofachievement level of the policy’s basic philosophy inthe citizens’ view. The suitability to the data of theone-factor model related to the cognitive appraisal
scale for policy was CFI > 0.999, RMSEA < 0.001(Figure 1). We assumed correlation between yA1 andyA2’s error variables.

Table 1 Distribution of Answers to Items Related toAchievement Level of Policy’s Basic Philosophy,WHPV,DPSURXGWKDWWKHHOGHUO\
ZLOOVSHQGWKHROGDJHLQ<<FLW\LVFRPIRUWDEOH[$
HQYLURQPHQWWRWKHHOGHUO\7KHHOGHUO\LQ<FLW\OLYH[$ KHDOWKLO\ERWKSV\FKRORJLFDOO\DQG
7KHHOGHUO\LQ<FLW\KDVD[$
IXQDQGIXOILOOLQJOLIH
XQLWSHRSOH [$&DWHJRULHV
QRWDWDOO QRWYHU\PXFK
DOLWWOH

YHU\PXFK     

QRWDWDOO&DWHJRULHV
DOLWWOH
TXLWH

      

7KHHOGHUO\LQ<FLW\OLYHEULJKWO\
DQGKDSSLO\E\HQKDQFLQJWKH[%
VRFLDOSDUWLFLSDWLRQVXSSRUWRI<
FLW\      

7KHHOGHUO\LQ<FLW\DUHVHQGLQJD
UHDVVXULQJROGDJHHYHQLIFDUHLV[%     
QHHGE\HQKDQFLQJWKHUHJLRQDO
OLIHVXSSRUWRI<FLW\ 

[%

7KHHOGHUO\LQ<FLW\DUHOLYHO\DQG
LQGHSHQGHQWOLYLQJE\HQKDQFLQJ     
WKHSUHYHQWLYHORQJWHUPFDUH
VHUYLFHRI<FLW\ <FLW\KDVEHFRPHDQ
HQYLURQPHQWWKDWIDFLOLWDWHVFDUH
DWKRPHE\HQKDQFLQJWKHORQJWHUPFDUHVXUYLFHRI<FLW\      

[%     XQLWSHRSOH           

yB1

Achievement
L l off Policy’s
Level
P li ’
Basic Philosophyn=244n= 244
ʖ2 =0.120
df =1
CFI>0.999
RMSEA<0.001
Estimator:WLSMV

.622†
.791
.993
.990

yA1
yA2

VXIILFLHQW

7KHHOGHUO\LQ<FLW\OLYHKHDOWKLO\
DQGOLYHO\E\HQKDQFLQJWKH[%KHDOWKSURPRWLRQVXSSRUWRI<
FLW\ProgressgofProgram

.510

yA3
yA4

*Thepathcoefficientinthefigureisastandardizedestimate.
*†showstheconstrainedpathformodelidentification.
py*Inordertoavoidcomplexityerrorvariableswerenotshown.

Figure 1 Factor Structure of Cognitive Appraisal Scale for Policy－－n=244n= 244
ʖ2 =2.395
df =4
CFI>0.999
RMSEA<0.001
Estimator:WLSMV

.813†
.898
.964
964
.952
.915

yB2

.626

yB3
B3
yB4
yB5

*Thepathcoefficientinthefigureisastandardizedestimate.
*†showstheconstrainedpathformodelidentification.
py*Inordertoavoidcomplexityerrorvariableswerenotshown.

Figure 2 Factor Structure of Cognitive Appraisal Scalefor Program社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

Table 3 Distribution of Answers to Items Related to Satisfaction of Needs by Project,WHPV+HDOWK3URPRWLRQ6XSSRUW
7KHVHUYLFHRQSURPRWLRQRIVHOIKHOSJURXSDQGFRPPXQLW\[& RUJDQL]DWLRQDFWLYLWLHVIRUKHDOWKSURPRWLRQLQ<FLW\VDWLVILHV
WKHQHHGVRIFLWL]HQV&DWHJRULHV
DOLWWOH
TXLWH

QRWDWDOO

VXIILFLHQW

















[&

7KHVHUYLFHRQHQKDQFHPHQWRISUHYHQWLRQRIOLIHVW\OHUHODWHG
GLVHDVHVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[&

7KHVHUYLFHRQPHDVXUHVDJDLQVWLQIHFWLRXVGLVHDVHVRIWKH
HOGHUO\LQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















6RFLDO3DUWLFLSDWLRQ6XSSRUW['

7KHVHUYLFHRQPDNLQJOLYLQJWKURXJKFRPPXQLW\LQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















['

7KHVHUYLFHRQVXSSRUWIRUDFWLYLWLHVRIHOGHUO\FOXEVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















['

7KHVHUYLFHRQSURPRWLQJVXSSRUWHDFKRWKHULQWKHUHJLRQLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















['

7KHVHUYLFHRQH[SDQVLRQRIHPSOR\PHQWRSSRUWXQLWLHVIRUWKH
HOGHUO\LQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















5HJLRQDO/LIH6XSSRUW[(

7KHVHUYLFHRQSURPRWLRQRIHOGHUO\ZHOIDUHSURMHFWLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQSURPRWLRQRIDUELWUDU\SURMHFWLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQSURPRWLRQRIKHDOWKDQGZHOIDUHSURMHFWLQ<
FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQHQKDQFHPHQWRIFRPPXQLW\JHQHUDOVXSSRUW
FHQWHULQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQVWUHQJWKHQFRRSHUDWLRQZLWKORFDOUHVLGHQWVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

































7KHVHUYLFHRQVWUHQJWKHQFRRSHUDWLRQZLWKUHODWHG[( RUJDQL]DWLRQVLQWKHGLVWULFWLQ<FLW\VDWLVILHVWKHQHHGVRI
FLWL]HQV[(

7KHVHUYLFHRQHQFRXUDJHGLVVHPLQDWLRQRIVXSSRUWIRUWKH
HOGHUO\ZLWKGHPHQWLDLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQFRQVWUXFWLRQRIVXSSRUWV\VWHPIRUWKHHOGHUO\ZLWKGHPHQWLDLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQHDUO\GHWHFWLRQDQGSUHYHQWLRQRIGHPHQWLDLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQH[SDQVLRQDQGVXSSRUWRIFDUHDQGZHOIDUHVHUYLFHVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[(

7KHVHUYLFHRQFRQVWUXFWLRQRIDVDIHV\VWHPLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















3UHYHQWLYH/RQJ7HUP&DUH6HUYLFH[)

7KHVHUYLFHRQSURPRWLRQRISULPDU\SUHYHQWLRQWDUJHWSURJUDPVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















[)

7KHVHUYLFHRQSURPRWLRQRIVHFRQGDU\SUHYHQWLRQWDUJHWSURJUDPVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

















































































































































7KHVHUYLFHRQVXSSRUWDQGHQKDQFHPHQWRISUHYHQWLYHORQJ
[) WHUPFDUHPDQDJHPHQWLQ<FLW\VDWLVILHVWKHQHHGVRI
FLWL]HQV(QKDQFHPHQWRI/RQJ7HUP&DUH6HUYLFH
7KHVHUYLFHRQHQKDQFHPHQWRILQKRPHVHUYLFHVFHQWHULQJRQ[* FRPPXQLW\EDVHGVHUYLFHVLQ<FLW\VDWLVILHVWKHQHHGVRI
FLWL]HQV
7KHVHUYLFHRQLPSURYHPHQWRIIDFLOLW\VHUYLFHVVXFKDVORQJ
[* WHUPFDUHLQVXUDQFHIDFLOLWLHVLQ<FLW\VDWLVILHVWKHQHHGVRI
FLWL]HQV
7KHVHUYLFHRQSURPRWLRQDQGHQKDQFHPHQWRIORQJWHUPFDUH[* LQVXUDQFHEHQHILWFRVWRSWLPL]DWLRQSURMHFWLQ<FLW\VDWLVILHV
WKHQHHGVRIFLWL]HQV[*

7KHVHUYLFHRQLPSURYHDQGVHFXUHWKHTXDOLW\RIORQJWHUP
FDUHVHUYLFHVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV

7KHVHUYLFHRQIDLUPDQDJHPHQWRIVXUYH\DQGMXGJLQJRI[* FHUWLILFDWLRQRIORQJWHUPFDUHLQ<FLW\VDWLVILHVWKHQHHGVRI
FLWL]HQV
7KHVHUYLFHRQVWUHQJWKHQJXLGDQFHDQGVXSHUYLVLRQWR[* FRPPXQLW\EDVHGVHUYLFHRIILFHVDQGJXLGDQFHWRORQJWHUP
FDUHVHUYLFHRIILFHVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQV
7KHVHUYLFHRQVHFXUHUHFHLSWRIQXUVLQJFDUHLQVXUDQFH[* FKDUJHVDQGSURSHUH[HFXWLRQRILQVXUDQFHEHQHILWVLQ<FLW\VDWLVILHVWKHQHHGVRIFLWL]HQVXQLWSHRSOH －－社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

The suitability to the data of the second-order factor

primary factor, and the satisfaction of needs by projectmodel related to the cognitive appraisal scale foris a secondary factor. Its suitability to the data wasproject was CFI = 0.984, RMSEA = 0.088 (Figure 3).

CFI = 0.983, RMSEA = 0.067 (Figure 4). In addition, we

3. R elationships between Cognitive Appraisalsof Elderly Welfare Policy, Program, andProject for the Elderly at Home.

confirmed a statistically significant positive relevance

We assumed an analysis model in which theand secondary factor (path coefficient: 0.739).

between the dependent variable and primary factor(path coefficient: 0.862) and between the primary factor

achievement level of policy’s basic philosophy is a

The contribution ratio toward the achievement level

dependent variable, the progress of program is aof policy’s basic philosophy in the model was 70.5% .

.962†
.990
990
.934
HealthPromotion
Support

.961†
961†
.896
.892
.905
.945†
.927
.920
.859
.912
.911
.932
932
.967
.906
.937
.926
.938†
938†
.941
.867

SocialParticipation
Participation
Support

.869†
869†
.913
SatisfactionofN d b
Needsby
Project

RegionalLife
g
Support

.981
981
.953

n=244n244
ʖ2 =1002.441
df =345
CFI=0.984
RMSEA=0.088
Estimator:WLSMVPreventiveLongͲTermCare
Service.866

Enhancement
ofLongͲTerm
CareService

.948†
948†
.961
.959
.973
.871
.897
897
.941

*Thepathcoefficientinthefigureisastandardizedestimate.
*The
path coefficient in the figure is a standardized estimate*†showstheconstrainedpathformodelidentification.
*Inordertoavoidcomplexityerrorvariableswerenotshown.

yC1
yC2
yC3
C3
yD1
yD2
yD3
yD4
yE1
yE2
yE3
yE4
yE5
yE6
yE7
yE8
yE9
yE10
yE11
yF1
yF2
yF3
yyG1
yG2
yG3
yG4
yG5
yG6
yG7

Figure 3 Factor Structure of Cognitive Appraisal Scale for Project

HealthPromotion
SupportSocialParticipation
Participation
Support

sex

.880†
880†

age

.204
204

.119
119

.930
RegionalLife
g
Support

.971
971
.952

PreventiveLongͲTermCare
ServiceEnhancement
ofLongͲTerm
CareService.872

SatisfactionofN d b
Needsby
Project.739
739

R2 =0.043

ProgressofgProgram

R2 =0.606n=244ʖ2 =1435.136
df =688
CFI=0.983
RMSEA=0.067
Estimator:WLSMV

.862
862

Achievement
L l f P li ’
LevelofPolicy’s
BasicPhilosophy

R2 =0.705*Thepathcoefficientinthefigureisastandardizedestimate.
*† showstheconstrainedpathformodelidentification.
Inordertoavoidcomplexity,endogenousobservedvariables,error
order to avoid complexity, endogenous observed variables, error*Invariables,correlationbetweencontrolvariables,andinsignificantpathwerenotshown.

Figure 4 Relationships between Cognitive Appraisals of Elderly Welfare Policy, Program,and Project for the Elderly at Home－－

社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

Ⅳ . Discussion

in the study can be used for not only logic model

We conducted a study that aims to reveal theevaluation but also performance measurement as therelationships between cognitive appraisals of elderly

outcome index to measure the policy improvement

welfare policy, program, and project for the elderly at

effect. In addition, as in the study, disclosing thehome, with the intention of obtaining basic required

validity of the indices used for policy evaluation

information that contributes to the systematization

becomes necessary and important information forof policy evaluation. Concretely, we assumed a causal

considering the policy evaluation method.

model that positions cognitive appraisals of elderly

Second, we revealed that the causal model

welfare policy, program, and project by the elderly

assuming the logic model outcome according to

at home as outcome indices of the logic model andthe elderly welfare policy structure is statisticallyconducted a logic model evaluation to be examinedsupported. In the model, the cognitive appraisal of

in SEM. Conventional policy evaluation studies point

projects in the citizens’ view affects the cognitive

out the weak points of statistical tests based on theappraisals of policy through the cognitive appraisals ofas aprograms. Furthermore, we revealed that each policy,

quantitative grasping method of policy effects andprogram, and project evaluation has a statisticallyexperimental or quasi-experimental design

9)Furthermore,significant positive relevance. We think the result that

we adopted WLSMV for the estimator since theempirically revealed the appropriateness of the impact

observation variables of the scale for the analysis aretheory of elderly welfare policy and the contribution

the ordinal scale. WLSMV corrects the standard error

to the achievement of the policy’s basic philosophy

according to the distribution of data and outputs ain the view of citizen by the elderly welfare policy

stable estimate regardless of the sample size, thestructure. In addition, we interpret the result as

number of observation variables, and the normality

implying that there is no program and project thatraise the introduction of SEM.

3)23)24)

We believe it was appropriate todoes not affect the policy’s basic philosophy because

adopt SEM by WLSMV for the statistical analysis of

the contribution ratio and the pass coefficient for thethis study.

cognitive appraisal for policy obtained in the study

25)

of latent variables.

In the study, we first revealed that the following

were not low. In Japan, guidelines state that policies

are in the statistical tolerance regions: the construct

are to be evaluated from the basic three perspectives

validity from the structural aspect26)27) of the cognitive(necessity, efficiency, and efficacy).28) Depending on

appraisal scale for policy consisting of four items,

the nature of the policy, two further perspectives are

the cognitive appraisal scale for program consisting

added (fairness and priority).28) We believe the logic

of five items, and the cognitive appraisal scale formodel evaluation satisfies the above five perspectives

project consisting of 28 items. In recent years, an

and that a certain knowledge is obtained as a result.

examination of the validity of indices used for policy

For example, the necessity can be examined byevaluation in Japan has been required because

the presence or absence of statistically significant

conventional policy evaluations have conducted many

relevance between the components of the logic model;

performance measurements that rely on indicesthe efficiency can be examined by the contribution

whose validity has not been sufficiently examinedratio to the outcome and impact per resource whenThe

comparing the input of the policy with the path

results of the study indicate that a certain statisticalcoefficients of the considered logic model; the efficacy

affirmation was obtained for the validity of the scales

can be examined by the suitability to the data of theused in the study, and each scale has conceptual

developed logic model; the fairness can be examinedunidimensionality. This is the evidence for scoringby classifying the beneficiaries of policy by attributes

policy outcomes at the unit of factors and for using

and conducting multi-group analysis etc.; and thethe score for comparison. We believe the three scales

priority can be examined by comparing the suitabilityor whose outcomes cannot be compared.

7)

－－

社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

to the data of the logic model having common policy

Internal Affairs and Communications. [retrieved

beneficiaries and input, the presence or absence andon 1 Apr 2017]. Retrieved from the Internet:

strength of relevance between variables, and the<URL: https://www.env.go.jp/guide/seisaku/

contribution ratio to outcome and impact. From thecommi/pdf/24_1_5.pdf> (in Japanese)

above, we believe the method of evaluating the logic

3) Rossi PH., Lipsey MW., Freeman HE. Evaluation:model using the SEM proposed in the study is aligned

A Systematic Approach, Seventh Edition. Sage

with the perspective of conventional policy evaluation.

Publications, 2003.

Furthermore, we believe the method of evaluating

4) Final Report on the Policy Evaluation System.

the logic model using the SEM proposed in the study[online] Dec 2000, Study Group on Policy

satisfies the five perspectives of policy evaluation

Evaluation Method. [retrieved on 1 Apr 2017].

as well, and detailed information can be obtained

Retrieved from the Internet: <URL: http://www.

compared with conventional policy evaluation

soumu.go.jp/main_sosiki/hyouka/s_saihou.htm>

focusing on the performance measurement.

(in Japanese)

Therefore, in the study, we proposed the logic

5) Institute of Administrative Management. Detailed

model evaluation method using citizen index with

Policy Evaluation Guidebook: comprehensive

validity, which was not seen in the conventionalcommentary on laws, basic policies andsurvey, and examined it empirically. In addition,

guidelines. Gyosei, 2008. (in Japanese)

we revealed the possibility and importance of the6) Yamaya K. Policy Evaluation. Minerva-Shobo,

proposed method as well as the knowledge that

2011. (in Japanese)

contributes to the achievement of the policy’s basic

7) Ono T. Performance Measurement in thephilosophy. In the future, further study of the policy

Evaluation Practices of Japanese Government:

evaluation method is desirable. Concurrently, it is

Exploring the Quality of Measurement. Japanesenecessary to try to cross-validate the proposed logic

Journal of Evaluation Studies. 2013; 13(2): 21-36.

model, evaluate the whole logic model not limited

8) Logic model Development Guide. [online] 2 Feb

to the impact theory, introduce administrative

2006, W.K. Kellogg Foundation. [retrieved on 1

indices, and consider external factors and regional

Apr 2017]. Retrieved from the Internet: <URL:

differences and their application to the evaluation

https://www.wkkf.org/resource-directory/of policies, such as community care and community

resource/2006/02/wk-kellogg-foundation-logic-

development.

29)

However, we believe the findings

model-development-guide>

obtained in the study will have significant implications

9) Ryu Y, Sasaki R. Theory and Technique of“Policy

for the systematization of policy evaluations and

Evaluation”: Revised Edition. Taga-Shuppan,

project evaluations,30)

penetration of the concept of

performance measurement,

31)

2004. (in Japanese)and the future direction

10) Lee Y. The Use of Program Theory Evaluationof social welfare policies.

in Welfare: Information That Will Contribute to
the Improvement of the Program. Social science

References

review. 2016; (118): 1-12.

1) Basic Guidelines for Implementing Policy11) Nishide J. A New Current of Evaluation for

Evaluation. [online] 23 Dec 2001, Ministry ofManagement in Local Government Reform:

Internal Affairs and Communications. [retrieved

A Shift from Performance-focused to Theory-on 1 Apr 2017]. Retrieved from the Internet:

focused Evaluation Perspective. Japanese Journal<URL: http://www.soumu.go.jp/main_sosiki/of Evaluation Studies 2005; 5(1): 15-25.

hyouka/hyoka_hosinhonbun.html> (in Japanese)

12) Sato T. Studies on regional policies: A proposal

2) Guidelines for Implementation of Policyof the method to use the logic models on setting

Evaluation. [online] 16 Dec 2005, Ministry of

the indicators to evaluate the policies and－－社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017

programs at local government: through a case25) Kosugi K, Shimizu Y. Introduction to Structural

study on Toyonaka Municipal Government. The

Equation Modeling with Mplus and R. Kitaohji-

Japanese journal of regional policy studies. 2003;

Shobo, 2014. (in Japanese)(1): 147-153.

26) Messick S. Foundations of Validity: meaning and13) Sato T. About the method evaluating the activity

consequences in psychological assessment. ESTof council of social welfare: the logic model's

Research Report. 1993; 2: 1-18.

application in program evaluation. Kansai

27) Shimatani H. Granting Course Credit for

University of Social Welfare. 2010; 14(1): 45-52.

Performance on English Proficiency Tests:14) Sato T. About Project Evaluation of Municipal

Validity and Washback Effects. KumamotoSocial Welfare Council: Utilization of Logic Model

University Repository System. 2013; 62: 91-90.

by Program evaluation. Social Worker 2012; (19):28) Standard Guidelines on Policy Evaluation. [online]

8-17. (in Japanese)

15 Jan 2001, Ministry of Internal Affairs and15) Minamoto Y. Practice of Theory Evaluation in

Communications. [retrieved on 1 Apr 2017].

Japanese Policy Evaluation Setting: Applicability

Retrieved from the Internet: <http://www.

of Practitioners Co-evaluation and Reflection

soumu.go.jp/main_sosiki/hyouka/gaido-

Model. Governance Studies. 2013; (9): 17-30.

gaidorain1.htm> (in Japanese)16) Yonehara A. Utilization of Social Survey in29) Dei R, Mihara T, Mikane S, et al. Relationship

Theory Evaluation: A Case Study of "Co-between the Status and the Satisfaction Level

productive Social Survey". Governance Studies.

for Local Quality-of-Life for the Elderly Living at

2015; (11): 173-188.

Home. Japanese Journal of Social Welfare. 2015;17) Bradshaw J. A taxonomy of social need,56(2): 75-87.

McLachlan, G. ed. Problems and Progress in30) Oshima I. Present situation and its social

Medical Care: essays on current research,

conditions that inhibit evidenced-based practices

seventh series. Oxford University Press, 1972.

and their formative approaches from established

18) Lazarus RS., Folkman S. Stress, Appraisal, andin the Japanese society: Implications for

Coping. Springer Publishing Company, 1984.

improvement. Japanese Journal of Evaluation19) Miura F. Social Welfare Policy Research: Welfare

Studies. 2014; 14(2): 17-28.

policy and Welfare reform, Revised Edition.

31) Tanabe S. The Role of Program Evaluation as

National Social Welfare Council, 1997. (inComplement to Performance Measurement: The

Japanese)

Case of GPRAMA in the United States. Japanese20) Muthen LK., Muthen BO. Mplus User's Guide:

Journal of Evaluation Studies. 2014; 14(2): 1-16.

Seventh Edition. Los Angeles, 2012.
21) Bentler PM. Comparative Fit Indexes in
Structural Models. Psychological Bulletin. 1990;
107: 238-246.
22) Steiger HH., Lind JM. Statistically Based Tests
for the Number of Common Factor. annual
meeting of the Psychometric Society, Iowa City,
1980.
23) Weiss C H. Evaluation: Methods for Studying
Programs and Policies, Second Edition. Prentice
Hall, 1997.
24) Yasuda T, Watanabe N. Method of Program
Evaluation Study. Shinyo-Shya, 2008. (in Japanese)－－社会医学研究．第 34 巻 2 号．Bulletin of Social Medicine, Vol.34(2）2017－ 10 －