Melco Management Accounting Research
Discussion Paper Series
No.MDP2017-001
Combined Effects of Subjective Performance Evaluation, Environmental
Uncertainty, and Budget Rigidity on Financial Performance

January 2017

Keita Masuya*
Graduate School of Business and Commerce, Keio University
2-15-45 Mita, Minato-ku, Tokyo
108-8345, Japane-mail:masuya-keita@keio.jp
Eisuke Yoshida
Faculty of Business and Commerce, Keio University
2-15-45 Mita, Minato-ku, Tokyo
108-8345, Japane-mail:eyoshida@fbc.keio.ac.jp
Zhiming Xu
Graduate School of Business and Commerce, Keio University
2-15-45 Mita, Minato-ku, Tokyo
108-8345, Japane-mail: zhiming.xu@keio.jp*Corresponding author

The Melco Foundation
Nagoya, Japan
Discussion Paper Series of the Melco Management Accounting Research, in order to promote
the study of management accounting, has published this unfinished paper onto the Web. Pleaseobtain the permission of the author when citing this paper.

Combined Effects of Subjective Performance Evaluation,
Environmental Uncertainty, and Budget Rigidity
on Financial Performance

Keita Masuya*
Graduate School of Business and Commerce, Keio University, Japan
E-mail address: masuya-keita@keio.jp

Eisuke Yoshida
Faculty of Business and Commerce, Keio University, Japan
E-mail address: eyoshida@fbc.keio.ac.jp

Zhiming Xu
Graduate School of Business and Commerce, Keio University, Japan
E-mail address: zhiming.xu@keio.jp*Corresponding author

Abstract
This study examines the combined effects of subjective performance
evaluation, environmental uncertainty, and budget rigidity on financial
performance. Specifically, this study assumes that the beneficial role ofsubjective performance evaluation in highly uncertain environment will
decrease because high budget rigidity causes dysfunctional behavior inachieving targets. Using survey and archival data for 252 firms, the results
support the hypothesis that subjective performance evaluation enhancesfinancial performance as environmental uncertainty increases and budget

1

rigidity decreases. In addition, supplementary analysis indicates such
positive performance effects of subjective performance evaluation become
more prominent when performance measure diversity is low. Thesefindings suggest that the effectiveness of subjective performance
evaluation is determined by congruence not only with environmental
uncertainty, but also with budget rigidity. Furthermore, this study
empirically indicates that under certain situations, it is rational for there to
be a substitutional relationship between subjective performance
evaluation and performance measure diversity for performance
improvement.

Keywords
Subjective performance evaluation, Environmental uncertainty, Budget rigidity,
Performance measure diversity, Financial performance, Questionnaire survey

1 Introduction
Managerial performance evaluation is a central means to achieve goal
congruence between firms and managers. In order to align their interests, which are
not always consistent, it is necessary to set performance targets that contribute toachieving organizational objectives and provide incentives for direct effort to achievegoals. Formal performance evaluation provides such incentives through setting
performance targets, weighting them, and rewarding their achievement (Merchant &Van der Stede, 2012).
However, there are some limitations for performance evaluation based on formal
rules. For example, risks exist that cause distorted incentives. Quantitatively
evaluating important behavior, such as knowledge creation or cooperation, is notalways easy. Nevertheless, evaluation based only on specific performance measures
might provide managers with distorted incentives to commit to measured targets while
ignoring other relevant behavior (Holmstrom & Milgrom, 1991). In addition, there is2the risk of unexpected poor performance by managers. Uncontrollable factors, such as
economic crisis or organizational interdependencies, influence the achievement of
performance targets. In this situation, performance evaluation based solely on theachievement of targets enhances risk-bearing problems (Holmstrom, 1979). Becauseof these limitations, performance evaluation based on formal rules might lower amanager’s motivation or enhance unintended dysfunctional behavior that impedes theachievement of organizational objectives.
Subjective performance evaluation has been considered as an effective way to
overcome these limitations. Subjectivity is used to evaluate managerial performancebased not on formal rules but on subjective perception or judgement of evaluators.
Previous research provides three types of usage of subjectivity in performance
evaluation: incorporating qualitative non-financial measures, using subjectivity in expost weighting of performance measures, and allowing ex post discretional
adjustments based on factors other than specified performance measures (Bol, 2008;
Gibbs, Merchant, Van der Stede, & Vargus, 2004; Hoppe & Moers, 2011).
Some studies deal with the performance effects of subjective performance
evaluation. Specifically, they show that subjective performance evaluation promotes
value-enhancing behavior, such as knowledge creation or cooperation and enhances
motivation through reducing risk-bearing problems (Bol & Smith, 2011; Cheng &Coyte, 2014; Gibbs et al., 2004; Hoppe & Moers, 2011). Furthermore, some studies
examine the combined effects of subjective performance evaluation and contextual
factors, such as environmental uncertainty, business unit strategy, and trust onorganizational performance (Gibbs et al., 2004; Govindarajan, 1984; Govindarajan &Gupta, 1985).
It can be assumed that not only these contextual factors but also higher pressureof financial targets substantially influences the performance effects of subjective
performance evaluation. Evaluation standards or criteria are not clear for managers(evaluatees) when their performance is subjectively evaluated because such evaluationaccompanies the evaluator’s own preferences or interpretations (Bol, 2008; Luft,
Shields, & Thomas, 2016; Merchant, 1989; Ross, 1995). In this situation, heighteningpressure of specific financial targets leads to managers’ perceptions that achievementof targets is critical (Marginson & Ogden, 2005). Managers who perceive theimportance of achieving financial targets might take dysfunctional behavior, such as
manipulation of performance measures, like pulling profits from future periods, into
the current period to achieve targets (Merchant, 1990). Furthermore, the risks ofderiving this dysfunctional behavior are enhanced when the external environment
becomes more uncertain because the causal relationship between managerial behavior

3and results becomes unclear (Govindarajan, 1984; Merchant, 1990). Subjective
performance evaluation is suitable for eliminating this dysfunctional behavior throughex post adjustment in performance evaluation (Bol, 2008); however, its beneficial
effects might be offset by higher pressure of financial targets. Hence, it can beassumed that performance effects vary by combination of subjective performance
evaluation, environmental uncertainty, and the pressure of financial targets, but thereis scarce empirical evidence about their interaction effects.
Against this background, the aim of this study is to investigate the combined
effects of subjective performance evaluation, environmental uncertainty, and thepressure of financial targets on financial performance. Specifically, this study focuseson budgetary targets as the representative position in financial targets and considers
the effects of budget rigidity as a notion that reflects the pressure of financial targets.
The results from survey and archival data for 252 samples of large Japanese firms
show evidence consistent with expectations. That is, subjective performance
evaluation enhances financial performance (both return on assets, ROA, and sales
growth rate) as environmental uncertainty increases and rigidity of budgetary targets
decreases. Furthermore, supplementary analysis indicates that positive performance
effects of subjective performance evaluation appear more prominently when
performance measure diversity is low.
This study makes several contributions to the related literature. First, this study
finds specific conditions in which subjective performance evaluation enhances
organizational performance. Previous research on subjective performance evaluation
empirically indicates that contextual factors, such as environmental uncertainty,
business unit strategy, and trust, determine the effectiveness of subjective performance
evaluation on organizational performance (Gibbs et al., 2004; Govindarajan, 1984;
Govindarajan & Gupta, 1985). The current study reveals that not only these contextual
factors but also the rigidity of budgetary targets determines the effectiveness ofsubjective performance evaluation. In addition, supplementary analysis indicates thatthe performance effects of subjective performance evaluation under specificconditions become prominent when performance measure diversity is low.
Second, this study finds effects of subjective performance evaluation on
financial performance. One subject of the performance evaluation literature is whether
outstanding performance evaluation theoretically enhances financial performance(Franco-Santos, Lucianetti, & Bourne, 2012; Ittner, Larcker, & Randall, 2003). The
results of this study contribute to these streams of literature by clarifying the effects offinancial performance in certain situations.

4Finally, the results of this study suggest the importance of empiricallydistinguishing between two notions: performance evaluation style and budget rigidity.
Research on reliance on accounting performance measures (RAPM) uses two terms,
performance evaluation styles (objective, subjective, and both) and budget rigidity,
without clear distinction. Some research determines the cause of the inconsistency of
empirical results or theory–practice gaps on undisposed notions (Hansen, Otley, &Van der Stede, 2003; Libby & Lindsay, 2010; Otley & Fakiolas, 2000). Given these
problems, the empirical results of this study, which clearly distinguish between two
overlapping notions, show that financial performance effects vary depending on the
combination of subjective performance evaluation and budget rigidity, even in thesame situation.
The remainder of this paper is structured as follows. Section 2 reviews related
prior literature on subjective performance evaluation and budget rigidity, and develops
the hypothesis of the study. Section 3 explains the data collection and variablemeasurement. Section 4 presents the results of analysis. Finally, section 5 shows thecontributions, limitations, and directions for future research.

2 Literature review and hypothesis development
This section reviews the previous literature on two key concepts: subjective
performance evaluation and budget rigidity. Specifically, the literature review mainly
focuses on the relationship between each concept and environmental uncertainty.
Finally, a hypothesis is developed regarding the effects of the contingency relationshipbetween environmental uncertainty, subjective performance evaluation, and budget
rigidity on financial performance.

2.1 Environmental uncertainty and subjective performance evaluation
Some studies focus on managers’ controllability and environmental uncertainty
as contextual factors that influence subjectivity in performance evaluation (Bol &Smith, 2011; Gibbs et al., 2004; Hoppe & Moers, 2011; Woods, 2012). These studies
assume that subjectivity in performance evaluation is used in order to enhance
adaptive behavior in highly uncertain environments or to reduce the risks of managers
performing poorly owing to uncontrollable factors. Gibbs et al. (2004) assume thefollowing two relationships. The first is that the more uncontrollable factors influencequantitative performance measures, the more subjectivity in performance evaluation

5

increases. The second assumption is that the more the external environment becomes
unpredictable or competitive, the more subjectivity in performance evaluation
increases. Likewise, Bol and Smith (2011) and Woods (2012) empirically find that theevaluation outcomes of managers are subjectively adjusted upward when they perform
poorly owing to uncontrollable factors. Furthermore, Hoppe and Moers (2011) focus
on subjective weighting of performance measures and presume a relationship withenvironmental uncertainty as follows. They assume that the ex ante weighting of
performance measures increases the risks of enhancing non-adaptive behavior tochanges in the environment when it is difficult to clarify optimal behavior that would
contribute to firms’ value ex ante, although managers have to adapt flexibly to changes
in the environment. Subsequently, the authors empirically confirm that the moreenvironmental unpredictability increases, the more is subjectivity in the weighting of
performance measures used to enhance adaptive behavior.
Alongside the development of the abovementioned studies, a few studies
examine the combined effects of environmental uncertainty and subjective
performance evaluation on organizational performance. Govindarajan’s (1984) study
based on contingency theory finds that superior organizations that perceive higher
environmental uncertainty use subjectivity in performance evaluation more.

2.2 Environmental uncertainty and budget rigidity
This subsection reviews the related literature on budget pressure. At first, somestudies focus on tightness (slack) of budgetary targets 1. Merchant and Manzoni (1989)
use field investigations to reveal the rationality of setting highly achievable budgetary
targets that contain slack in highly uncertain environments. In highly uncertain
environments, it is difficult to motivate managers to achieve very difficult targets
because uncontrollable factors hamper their achievement. In addition, greater
emphasis on achieving budgetary targets enhances the risks of deriving dysfunctional
behavior, such as manipulation of performance measures or restriction of discretionary
expenditures. While very difficult budgetary targets in highly uncertain environments
cause such serious problems, highly achievable targets met through substantial effort
motivate managers by lowering the effects of uncontrollable factors and by clarifying
1

Dunk and Nouri (1998, p.73) define budgetary slack as “the intentional
underestimation of revenues and productive capabilities and/or overestimation of costs
and resource.”6

the causal relationship between managers’ behavior and their results. Furthermore,
highly achievable targets that contain slack enhance adaptive behavior to external
environments that change on a short-term basis and increase discretionary
expenditures through enhancing operational flexibility2. Merchant and Manzoni (1989)
reveal the effectiveness of setting budgetary targets that contain slack to enable
adaption to changes in the environment or achievement of multiple performancemeasures, such as customer responsiveness (Davila & Wouters, 2005; Todd &Ramanathan, 1994).
The abovementioned studies reveal the effectiveness of budgetary slack fororganizational processes, but there has not been sufficient accumulation of empiricalevidence about their performance effects. Furthermore, some studies show an
inverted-U relationship between budgetary slack and organizational performance(Nohria & Gulati, 1996). Hence, it is difficult to explore optimal levels of budgetaryslack (tightness) that contribute to superior performance.
This study focuses on the rigidity of budgetary targets that influence theformation of budgetary slack. According to Sponem and Lambert (2016), budget
rigidity has two distinct design characteristics: budget revision (possibility of changing
the initially set budget) and budget re-forecast (existence of a forecast during theyear)3. The more the rigidity of budgetary targets increases, the more the budget
2

In addition, Merchant and Manzoni (1989) find that business units set very difficult
budgetary targets when their priorities are to obtain short-term profits. In this situation,
managers make decisions to restrict discretionary expenditure that would hamper
future performance in order to achieve short-term budgetary targets.
3
However, it cannot be said that there is consensus about the definition of budget
rigidity. For example, RAPM research defines budget rigidity regarding the extent towhich managers are evaluated based on the achievement of budgetary targets(Govindarajan, 1984; Hopwood, 1972). Specifically, these studies assume that budget
rigidity is high when managers are evaluated based on budgetary targets only. On thecontrary, budget rigidity is low when managers are evaluated based on a combinationof achievement of budgetary targets and other information. Hence, these studies
assume budget rigidity and performance evaluation as a similar concept. On the other
hand, recent studies reveal practices that combine budget rigidity and performance
evaluation. For example, Libby and Lindsay (2010) find that about half of respondent
firms emphasize the achievement of budgetary targets, but just 15 percent ofrespondents evaluate achievement only. Moreover, the authors find that subjective
performance evaluation is spread to some degree. In response to Libby and Lindsay’s

7revision or budget re-forecast decreases. If budget rigidity is high, the possibility ofcontaining slack by subordinates decreases because superiors intervene and control theslack-creation behavior of subordinates (Dunk, 1993; Merchant, 1985; Van der Stede,2000).
Prior studies reveal that a combination of environmental uncertainty and budget
rigidity strongly influences managerial behavior (Merchant, 1990; Van der Stede,2000). Merchant (1990) empirically finds that the positive effects of budgetary target
pressure on the manipulation of performance measures become more prominent whenenvironmental uncertainty is high. This is because a highly uncertain environment
makes the causal relationship between managerial behavior unclear and managers
perceive obscurity about how to achieve budgetary targets. As a result, the risks ofdysfunctional behavior to achieve fixed budgetary targets increase. Likewise, Van derStede (2000) assumes that adoption of differentiation strategies that accompany
obscurity between managerial behavior and its results necessitates slack in order tocope with such uncertainty. His empirical results support his assumption that adoptionof differentiation strategies leads to creation of slack by lowering budget rigidity andultimately enhance long-term managerial orientation. Furthermore, Lillis (2002)
empirically finds that lowering budget rigidity enhances the possibility of achieving
quality or customer responsiveness measures, which are a trade-off for achieving
budgetary targets.

2.3 Effects on financial performance of contingency relationship between
environmental uncertainty, subjective performance evaluation, and budget rigidity
This subsection postulates the combined effects of environmental uncertainty,
subjective performance evaluation, and budget rigidity on financial performance.
In highly uncertain environments, subjective performance evaluation enhances
motivation or perceived fairness by lowering the risks of managers performing poorly
and by enhancing adaptive behavior to changes in the external environment (Bol, 2011;
Bol & Smith, 2011; Hoppe & Moers, 2011). Because of these advantages, subjective
performance evaluation is suitable for highly uncertain environments and ultimately
contributes to higher organizational performance (Govindarajan, 1984).
While subjective performance evaluation has several advantages in highly
uncertain environments, evaluation standards or criteria are unclear for managers(2010) findings and Sponem and Lambert’s (2016) definition, this study assumes
budget rigidity as emphasis of or fixation on budgetary targets.

8(evaluatees) when their performances are subjectively evaluated (Bol, 2008; Luft et al.,2016; Merchant, 1989; Ross, 1995). Because of this evaluation obscurity, greater
emphasis on achieving budgetary targets leads to clarifying managers’ perceptions that
their responsibility is to achieve budgetary targets (Marginson & Ogden, 2005).
Managers come to perceive the importance of achieving budgetary targets, but thecausal relationship between managerial behavior and results becomes unclear whenenvironmental uncertainty increases (Govindarajan, 1984). In this situation, themotivational effects of budgetary targets decrease (Merchant & Manzoni, 1989).
Furthermore, managers who perceive obscurity about how to achieve budgetary targets
might adopt dysfunctional behavior to cope with such uncertainty (Hopwood, 1972;
Merchant, 1990; Merchant & Manzoni, 1989). Although previous literature suggeststhat subjective performance evaluation is effective for restricting such behavior, it canbe presumed that the beneficial effects of subjective performance evaluation decrease.
Merchant’s (1990) results seem to be consistent with this assumption. Specifically,
Merchant (1990) finds no mitigating effects of superiors’ consideration that contain
subjectivity on the manipulation of performance measures caused by high pressure ofbudgetary targets4. Hence, it can be assumed that highly rigid budgetary targets cause
serious problems, even in situations in which subjectivity in performance evaluation ishigh because of the obscurity of evaluation standards or criteria. Furthermore, highly
rigid budgetary targets cause serious problems, as subjectivity in performance
evaluation increases and the external environment becomes more uncertain.
From the aforementioned discussion, it can be assumed that congruence amonghigh subjectivity in performance evaluation, highly uncertain environments, and low
budget rigidity is associated with higher financial performance. Thus, the following
hypothesis is proposed.
Hypothesis: Subjective performance evaluation enhances financial performance asenvironment uncertainty increases and rigidity of budgetary targets decreases.
Figure 1 shows the proposed conceptual framework of this study.

4

Merchant (1990) denotes this result as contrary to the assumptions of Otley (1978),which is a representative study of RAPM research.

9

Figure 1
Conceptual Framework
Subjective performance evaluation

Financial performance
Budget rigidity
uncertainty

Environmental uncertainty

3 Method
3.1 Data collection
Large Japanese firms were selected for the sample. This is because anevaluator’s subjectivity plays a role in performance evaluation or personnel
assessment in Japan. Specifically, personnel assessment in Japan includes not only
performance evaluation but also evaluation on personnel capabilities, such as planningability or ability to get things done. Personnel assessment includes evaluation on
managerial attitudes, such as sense of responsibility or aggressiveness. Personnel
assessment accompanies subjectivity because evaluation based on capabilities orattitudes is composed of qualitative performance measures (Endo, 1994). In addition,
Endo (1994) indicates that subjectivity is a characteristic of managerial performance
evaluation, especially regarding decisions about bonus allocation.
Data were collected by questionnaire survey and archival database. The
respondents of the mail survey are managers/directors of corporate planningdepartments5. Corporate planning departments are appropriate respondents, as theirmain functions in large Japanese firms include investigation and exploration of
appropriate budgeting and performance evaluation systems.
5

The survey instrument was mailed to one company per person based on name andaddress information from a database of D-VISTION series of Diamond Press. If amanager’s name could not be obtained, the questionnaire was sent to the
managers/directors of the corporate planning department without names. All
respondents were required to fill in their names and departments on a form.

10Questionnaires were sent out on November 4, 2014 to 1,822 firms that werelisted on the First Section of the Tokyo Stock Exchange. The questionnaire was
required to be sent back before November 28, 2014. In an attempt to increase theresponse rate, a once-off follow-up postcard was sent to those who did not respond by
the due date. These processes resulted in 308 responses (a 16.9 percent response rate,
including responses after the follow-up postcard). There were 252 final samples foranalysis after eliminating sample firms that did not use budgets (2 samples), that did
not use budgetary targets for performance evaluation of business unit managers (32samples), and that contained missing data in questions (22 samples). Appendix A
presents the details of survey samples.
Several tests are implemented in order to investigate non-response bias. First,
t-tests comparing non-respondents to respondents across organizational size based onsales and number of employees are implemented. The results show no statisticallysignificant differences between respondents and non-respondents for both (p> 0.10).
Next, differences in organizational size between early and late respondents are
examined. As a result, although sales of early respondent firms are larger than sales oflater firms (t= 2.25, p= 0.025), there are no meaningful statistical differences withregard to numbers of employees (p> 0.10). Finally, two tests are implemented toinvestigate the difference in return data for all survey constructs between early and late
respondents. First, based on the results of a chi-square test, no significant difference isconfirmed in the usages of budgeting and the performance evaluation based on budget
targets between early respondents and late respondents (p> 0.10). Second, based on the
results of the t-test, which investigated the differences of mean-value of 31 survey
questions between early respondents and late respondents, early respondents have ahigher mean value in two survey items than do late respondents, but no significant
differences are found in the averages of the other 29 survey questions (p> 0.10)6.
Overall, the results of these tests support the absence of significant non-response biasthat would affect the data for analysis.

6

Meaningful statistical differences are found in “evaluate quantitative non-financial

targets subjectively on the basis of change of conditions or explanation by manager”(t= 2.208, p= 0.028) and “evaluate budget targets subjectively on the basis of changeof conditions or explanation by manager” (t= 2.266, p= 0.024). However, the effectsizes of the mean differences (Hedges’ g) are 0.29 and 0.26, respectively. Thus, it canbe said that the mean differences of the two items are not serious problems because theeffect size is small.

11

3.2 Variable measurement
3.2.1

Subjective performance evaluation

It is difficult to state that the measurement items of subject performance
evaluation (SPE) have been established by previous studies. For example,
Govindarajan (1984) and Govindarajan and Gupta (1985) measure superiors’
evaluation styles with reference to whether performance evaluation and thedetermination of incentive bonuses rely on formulas or subjective approaches.
Respondents were asked to indicate percentages when bonuses were decided partly ina subjective manner and partly in a formula-based manner. Likewise, Gibbs et al.
(2004) measure the use of subjectivity in the assignment of awarding bonuses in twoways, including whether any subjective bonus is given and the percentage of totalcompensation-related subjective bonus earned. Libby and Lindsay (2010) measure theuse of budgets in performance evaluation by asking respondents to choose one of five
options, namely (1) actual financial performance rigidly compared to budget, (2)budget target adjusted for uncontrollable budget variances, (3) budget target adjusted
at year-end using the formula established at the beginning of the year, (4) budget target
adjusted subjectively at the end of the year, and (5) both budget target and othersubjective factors used.
In this study, SPE is measured with the following four items in reference to
Libby and Lindsay (2010) and Van der Stede, Chow, and Lin (2006). Specifically, SPE
is measured for each typology of performance measures. First, subjectivity based onbudgetary targets was asked in a form adopted from Libby and Lindsay’s (2010)question (4). Second, respondents were asked to rate subjectivity on both objective andsubjective non-financial measures. Finally, to represent the features and practices ofJapanese management style, an item that reflects the subjective approach and is related
to the use of personnel capabilities and attitudes is added. These four items aremeasured on Likert scales of 1–7, where 1 indicates “not at all” and 7 indicates“absolutely correct.”Principal component analysis of the four items extracted one factor. The variable
is measured using the mean value of four items. The value of Cronbach’s alpha for thisconstruct is 0.68, which is deemed an acceptable level (Hair, Anderson, Tatham, &Black, 1998).

3.2.2

Environmental uncertainty

12Environmental uncertainty (EU) is measured using questionnaire items,
following Govindarajan (1984). This is because decision-makers’ perceptions of
uncertainty in external environments have a significant impact on performance
evaluation style and budget rigidity (Govindarajan, 1984).
The questionnaire items of EU are measured adopted from previous studies. For
example, Widener (2007) measures strategic uncertainties, a similar concept to EU,
based on the following three dimensions: operation (the five items of diffusion of
proprietary knowledge outside the organization, scale effects, scope effects, input
costs, and internal production innovation), competition (the three items of productintroductions in adjacent industries, market tactics of competitors, and new industry
entrants), and technology (the two items of changes in product technology that affectthe relative cost/efficiency to user and new technology). Based on the widely used
instrument developed by Hoque’s (2004) eight items (production and information
technologies, customer demand/tastes/preferences, market activities of competitors,
industrial relations, suppliers’ actions, deregulation and globalization, government
regulation and policies, and economic environment), Ekholm and Wallin (2011) add
three items that reflect new competitors/competing products, development of raw
material markets, and development of labor market. In this study, EU is measured
using eight items that reflect three dimensions common in prior studies: internalbusiness processes, competition, and technology. These eight items are measured on
Likert scales of 1–7, where 1 indicates “very predictable” and 7 indicates “veryunpredictable.”
The results of principal component analysis of the eight items indicate two
factors. The second factor, which includes items of industrial relations, government
regulation, and policies that present high factor loading, is eliminated because thesefactors are influential factors on others. Hence, the first factor is used as a variable of
EU. The variable is measured using the mean value of six items that load on factor one.
Cronbach’s alpha for this construct is 0.82, which is a satisfactory level.

3.2.3

Budget rigidity

In this study, budget rigidity (BR) is defined as the emphasis of achievement ofbudgetary targets based on business-unit level. Many previous studies use an
instrument developed by Hopwood (1972) or Otley (1978) to measure BR, but it might
be better not to use their instruments because they are developed to investigate
practices at particular companies or business units (Otley & Fakiolas, 2000). In this13

study, BR is measured by Van der Stede’s (2000) seven items, which are developed fora cross-sectional survey and subsequently, are used by Libby and Lindsay (2010).
These seven items are measured on Likert scales of 1–7, where 1 indicates “not at all”
and 7 indicates “absolutely correct.” One item is eliminated because of the ceiling
effect. From the results of principal component analysis, one factor is abstracted. The
variable is measured using the mean value of six items. The value of Cronbach’s alphafor this construct is 0.84, which is an adequate level.

3.2.4

Financial performance

Financial performance is measured by ROA and 3-year sales growth rate
following related studies (Arnold & Artz, 2015; Ittner et al., 2003). Data are collectedfrom Nikkei NEEDS databases. ROA is measured by dividing earnings before
interests and taxes by total assets of the 2013 fiscal year multiplied by 100. The sales
growth rate is the rate of sales growth over 3 years in which the differences in thevalue of sales between the 2013 fiscal year and the 2011 fiscal year are divided by thesales of the 2011 fiscal year multiplied by 100.

3.2.5

Control variables

Several variables that impact financial performance should be controlled. First,
industry performance (INDUSTRY) is controlled. To assess industry performance, themean ROA of the 2013 fiscal year and the mean sales growth rate of 3 fiscal years on
the industry level are constructed. Each INDUSTRY measure is included in ROA andsale growth rates as dependent variables (INDROA, INDSGR)7.
Second, organizational size (SIZE) is included. According to the prior literature,organizational size influences performance evaluation and organizational performance
significantly (Bruns & Waterhous, 1975). In this study, organizational size ismeasured by the natural logarithm of total assets of the 2013 fiscal year, following
related studies (Arnold & Artz, 2015; Ittner et al., 2003). Data are collected from
Nikkei NEEDS databases.

7

The category of industry is based on the industrial classification and standard
industrial classification code formulated by the Securities Identification CodeCommittee of the Japan Exchange Group. Data are collected from the database “eol.”

14

Finally, the performance measure of diversity (DIVERSITY) should be controlled.
Previous studies demonstrate that DIVERSITY, including financial and non-financial
measures, might influence organizational performance prominently (Hoque & James,
2000; Ittner et al., 2003; Van der Stede et al., 2006). Moreover, questionnaire items of
SPE in this study are compatible with the typology of performance measuresdeveloped by Van der Stede et al. (2006). Because of this measurement, the financial
performance effects of SPE might contain noise caused by DIVERSITY. Hence,
DIVERSITY should be controlled. In this study, DIVERSITY is measured by eightitems. These eight items are measured on Likert scales of 1–7, where 1 indicates“emphasis not at all” and 7 indicates “emphasis extremely.” One item that measures
the importance of profit measures is eliminated because of the ceiling effect. The
variable is operationalized by a simple average of seven items, following related
studies (Ittner et al., 2003; Van der Stede et al., 2006). Cronbach’s alpha for thisconstruct is 0.80, which is satisfactory.
The descriptive statistics of the questionnaire items and results of principle
component analysis are presented in Appendix B.

4 Results
4.1 Descriptive statistics and variable correlation
Table 1 shows descriptive statistics and variable correlation. Panel B of Table 1
indicates that subjective performance evaluation and budget rigidity is positively
correlated (r= 0.117, p= 0.064). This result suggests that the relationship between
subjective performance evaluation and budget rigidity is not substitutional but also
complementary, as Libby and Lindsay (2010) suggest. Furthermore, correlation
between subjective performance evaluation and financial performance (ROA and sales
growth rate) is statistically significant (p> 0.10). In addition, this result reflects that itis difficult to find a simple linear relationship between subjective performance
evaluation and financial performance, as Govindarajan (1984) empirically shows.

15

Table 1
Descriptive statistics and variable correlation
Panel A: Descriptive statistics
Mean

S.D.

Min

Max

1

SPE

4.38

.95

1.00

7.00

2

EU

4.14

.76

2.00

6.17

3

BR

4.93

.85

2.00

7.00

4

INDROA

5.12

1.73

.13

9.96

5

INDSGR

13.10

6.27

−25.52

31.87

6

SIZE

11.91

1.66

8.30

17.45

7

DIVERSITY

4.80

.85

2.57

7.00

8

ROA

4.38

4.43

−3.08

37.02

9

Sales Growth Rate

13.97

20.64

−29.60170.40Panel B: Variable correlation
1

2

3

4

5

6

7

1

SPE

1

2

EU

−.137*

1

3

BR

.117✝

−.058

1

4

INDROA

.169*

−.020

.015

1

5

INDSGR.059−.105✝

−.026.347***

1

6

SIZE

−.071

−.077

−.007

−.493***

−.086

1

7

DIVERSITY.189**−.131*.388***

.073

.066

.052

1

8

ROA

.117✝

.179**−.065.382***.101−.409***.011

9

Sales Growth Rate

.079

.090

−.165**

.090

.082

−.119✝

−.056

8

9

.543***

1✝, *, **, and *** denote significance at the 10%, 5%, 1% and 0.1% levels, respectively (two-tailed).
Variable Definitions:
SPE= Subjective Performance Evaluation, EU= Environmental Uncertainty, BR= Budget Rigidity, INDROA= Industry ROA,INDSGR= Industry Sales Growth Rate, SIZE= Organizational Size, DIVERSITY= Performance Measure Diversity

4.2 Main analysis
In order to test the hypothesis, the following regression formula is estimated.

16

FINANCIAL PERFORMANCE = α+ β ₁ SPE+ β₂ EU+ β₃ BR+ β₄ SPE*EU+
β₅ SPE*BR+ β₆ EU*BR+ β7SPE*EU*BR+ ΣControls+ εSpecifically, three models are used to examine the hypothesis. Model 1 contains
each main exploratory variable, SPE, EU, and BR. Furthermore, INDUSTRY, SIZE,and DIVERSTY are included as control variables. Two-way and three-way interaction
terms are included in Models 2 and 3, respectively. For the hypothesis of the study to
be supported, the coefficients of the three-way interaction terms become negative,
because this study assumes that high BR decreases the positive performance effects of
SPE in high EU. It should be noted that interaction terms might produce a
multicollinearity problem. To avoid this problem, all independent variables are
centered and interaction terms are formed later.
Table 2 shows the estimation results on ROA and sales growth rate8. Initially, no
statistically significant relationship can be found between SPE and both financial
performance (ROA and sales growth rate) (p> 0.10, respectively). These results reflect
the difficulty of finding a linear relationship between them. Next, for Models 2a and 2b,which include two-way-interaction terms, negative coefficients of EU*BR on financial
performance are found (β= −0.879, p= 0.001, β= −2.933, p= 0.030, respectively). In
addition, the increases of the coefficients of determination from Models 1a and 1b are
statistically significant (ΔR²= 0.048, ΔF = 4.959, p= 0.002, ΔR²= 0.033, ΔF = 2.935, p=
0.034, respectively). These results reflect incongruence between high EU and high BR,
as primary studies indicate empirically (Merchant, 1990). Furthermore, Model 2bsuggests that the negative coefficients of SPE*BR are statistically significant (β=−0.436, p= 0.046). By taking into account the positive coefficients of SPE in Model 1a,
it can be assumed that high BR mitigates positive performance effects of SPE, as
expected. Finally, Models 3a 3b, which contain three-way-interaction terms among
SPE, EU, and BR, indicate that their coefficients are statistically significant, with thepredicted signs (β= −0.921, p= 0.000, β= −5.314, p= 0.000, respectively). Moreover,an increase of R² is significant (ΔR²= 0.061, ΔF = 22.566, p= 0.000, ΔR²= 0.094, ΔF =27.484, p= 0.000, respectively). Hence, it can be said that the combined influences ofSPE, EU, and BR on financial performance are prominent, as expected 9,10.
8

The sample for analysis is 252, but financial performance data on some firms cannot
be obtained from the database. Hence, the sample for the actual estimation decreased
from 252.
9
In order to confirm the robust stability of coefficients of the three-way-interaction
term in Models 3a and 3b, a non-parametric bootstrap procedure as the most common

17re-sampling method is applied (Wooldridge, 2013). Specifically, this test draws
random samples that are the same as the analysis shown in Table 2 from theabovementioned original survey data with 1,000 bootstrap replications (Efron &Tibshirani, 1986) for hypothesis testing and standard error estimation (Efron &Tibshirani, 1993). The results of the bootstrap approach indicate that the negative
coefficients of these three-way-interaction terms are statistically significant (β= −0.95,
p= 0.047, β= −5.32, p= 0.060, respectively) based on the bootstrap estimates ofstandard errors (SE= 0.478, SE= 2.829). The absence of the validation problems ofthese models is supported because the bootstrapped p-value and bootstrap estimatorreflect similar results of Models 3a and 3b in Table 2.
10
The questionnaire items for SPE include evaluation about managerial capability or
attitude, which is general in Japan. Because of this item, some criticism might arisethat the findings of this study are limited to Japanese firms. To generalize the resultsof this study for western companies, estimation of the regression formula, whichmeasures SPE from the other three items, is implemented. The results indicate no
prominent changes for performance effects of the three-way-interaction term in thecase of either ROA or sales growth rate (β= −0.899, p= 0.000, β= −4.68, p= 0.000).
These results give reasonable evidence that findings are not driven by Japanese uniquepractices and can apply to western companies.

18

Table 2
Effects of association between subjective performance evaluation,
environmental uncertainty, and budget rigidity on financial performance
Dependent

SPE

EU

BR

ROA
Model 1aModel 2aModel 3aModel 1bModel 2bModel 3b

.363

.297

.130

2.094

1.916

.960

(.255)(.249)(.242)(1.323)(1.311)(1.257).786**.731**.684**

1.984

1.770

1.535

(.250)(.245)(.235)(1.314)(1.301)(1.235)

−.415

−.422

−.556*−3.617*−3.610*−4.384**(.267)(.262)(.253)(1.404)(1.394)(1.331)

−.013

−.286

1.462

−.100

(.205)(.204)(1.081)(1.068)−.436*−.286−1.558−.697(.217)(.210)(1.148)(1.101)−.879***−.778**−2.933*−2.392✝

(.252)(.243)(1.342)(1.277)

SPE*EUSPE*BR

EU*BR

SPE*EU*BR

INDUSTRYSIZE

DIVERSITY

Intercepts

Sales growth rate−.921***−5.314***(.194)(1.014).567***.571***.569***

.259

.306

.349✝

(.167)(.164)(.157)(.211)(.210)(.199)−.757***−.742***−.682***−1.240−1.246−.866(.171)(.168)(.161)(.793)(.788)(.751)

.237

.203

.256

.097

.013

.283

(.272)(.269)(.258)(1.420)(1.419)(1.348)11.884***11.683***10.881***

N

25.347*25125.046*19.412*

248

R²

.248

.292

.353

.063

.097

.191

Adj. R²

.230

.266

.326

.040

.063

.157

4.959**

22.566***2.935*

27.484***ΔF✝, *, **, and *** denote significance at the 10%, 5%, 1%, and 0.1% levels, respectively(two-tailed). Estimated with ordinary least squares. Unstandardized coefficients are reported andstandard errors are in parentheses. VIF (variance inflation factor) is less than 1.3.

In order to understand the content of interaction, ad hoc analysis is implemented.
Following Aiken and West (1991), the regression line on financial performance when19

EU and BR take ±1SD, respectively, is estimated by using simple slope analysis.
Figure 2 shows the estimation results of four patterns of SPE’s regression line on
financial performance. The results indicate that SPE enhances financial performance
when EU takes high and BR takes low (β= 1.56, p= 0.000, β= 8.33, p= 0.000,
respectively). In addition, the results show that SPE decreases financial performance
when both EU and BR are high (β= −1.87, p= 0.000, β= −6.52, p= 0.012, respectively).
These results indicate that congruence among high SPE, high EU, and low BR is
associated with higher financial performance. Hence, the hypothesis of this study is
supported.

20

Figure 2
Effects of interaction between subjective performance evaluation, environmental
uncertainty, and budget rigidity on financial performance
Panel A: ROA as a dependent variable

Panel B: Sales growth rate as a dependent variable

4.3 Supplementary analysis
Some economic-based studies investigate the relationship between subjective
performance evaluation and performance measure diversity. Based on the21informativeness principle (Holmstrom, 1979), subjective performance evaluation
improves incentive contracts when it brings additional information about managerial
behavior (Baker, Gibbons, & Murphy, 1994). Similarly, using multiple performancemeasures is effective for the same reason (Feltham & Xie, 1994; Hemmer, 1996).
Regarding the relationship between both approaches, Hoppe and Moers (2011) findthat discretionary bonus allocation is used more in contracts based solely on financial
measures than in contracts that also include non-financial measures for risk-reduction
purposes. The authors interpret their results as follows: the use of multiple
performance measures is effective for risk reduction, but this effectiveness lowers thenecessity of using subjectivity in performance evaluation. Their results andinterpretations suggest a substitutional relationship between both approaches, but
empirical evidence about the performance effects of their substitutional relationship is
not enough. Hence, it is unclear that substitutional use is rational for performance
improvement.
In this study, supplementary analysis is implemented to examine whether the
performance effects of the three-way-interaction term vary by the degree of
DIVERSITY. Specifically, samples are divided by three groups in response to the scoreof DIVERSITY and the regression formula is estimated for each group11. Three groups
are divided at the score of DIVERSITY at below 25 percent, from 25 percent to 75
percent, and above 75 percent respectively12.
Tables 3 and 4 show the estimation results on financial performance for eachgroup. The results indicate that the regression line is not statistically significant for
medium and high degrees of DIVERSITY groups (p> 0.10). On the contrary, negative
coefficients of the three-way interaction term are statistically significant for the low
DIVERSITY group (β= −1.42, p= 0.000, β= −5.87, p= 0.002, respectively).

11

DIVERSITY is removed from the regression formula because this variable is used asa grouping variable.
12
The Score position of 25 percent is 4.2857 and that of 75 percent is 5.4286.

22

Table 3
Effects of association between subjective performance evaluation, environmental uncertainty,
and budget rigidity on ROA for each diversity group
DIVERSITY

Low
Model 2aModel 3aModel 1bModel 2bModel 3bModel 1cModel 2cModel 3c

.639

−.307

−.260

.003

.021

.018

.310

.280

.265

(.633)(.635)(.561)(.348)(.356)(.356)(.408)(.619)(.632)

1.780**

.800

−.131

.793*

.794*

.915*

−.243

.014

.031

(.615)(.632)(.607)(.335)(.338)(.355)(.396)(.519)(.533)

−.689

−.805

−.953✝

.235

.224

.128

−1.457**−1.384**−1.351*(.618)(.576)(.511)(.365)(.370)(.379)(.471)(.487)(.531)

.610

−.711

−.415

−.528

−.455

−.501

(.507)(.561)(.318)(.333)(.370)(.466)−.993✝

−.551

.009

.047

−.198

−.169

(.521)(.474)(.322)(.324)(.540)(.572)−1.360*−1.318**

.299

.180

−.244

−.225

(.537)(.474)(.500)(.511)(.519)(.536)BR

SPE*EUSPE*BR

EU*BR

SPE*EU*BR

INDUSTRY

SIZEIntercepts

High

Model 1aSPE

EU

Medium−1.417***

−.488

.105

(.362)(.440)(.632)

.314

.415

.168

.683**.682**.691**

.520✝

.560*

.557*

(.475)(.421)(.377)(.219)(.220)(.220)(.264)(.271)(.274)−1.049*−.741✝

−.760*−.704**−.654**−.642*−.732*−.672*−.670*(.449)(.405)(.358)(.242)(.247)(.247)(.257)(.267)(.270)

16.219*11.789✝

12.872*

10.545**9.922**

9.734**

12.691**

11.626**

11.614**

N

60

135

56

R²

.341

.515

.629

.249

.259

.267

.344

.366

.367

Adj. R²

.280

.438

.562

.220

.212

.214

.278

.259

.243

6.068

15.350

.580

1.232

.561

.028

ΔF

✝, *, **, and *** denote significance at the 10%, 5%, 1%, and 0.1% levels, respectively (two-tailed).
Estimated with ordinary least squares. Unstandardized coefficients are reported and standard errors are in parentheses.
VIF is less than 2.6.

23

Table 4
Effects of association between subjective performance evaluation, environmental
uncertainty, and budget rigidity on sales growth rate for low diversity group
DIVERSITY

Low
Model 4

SPE

EU

BR

Model 5Model 64.273

−1.288−1.197(3.310)(3.136)(2.882)8.072*

4.336

.473

(3.248)(3.152)(3.138)−4.610−5.039✝

−5.596*(3.257)(2.904)(2.674)5.940*.581(2.529)(2.865)−6.750*−4.885✝

(2.590)(2.450)−4.497*−4.292✝

(2.671)(2.455)

SPE*EUSPE*BR

EU*BR−5.873**

SPE*EU*BR(1.837)
INDUSTRY

SIZEIntercepts

−.020

.196

.286

(.505)(.429)(.395)

−.482

1.107

1.680

(1.783)(1.553)(1.438)−2.50220.202

N−12.181

59

R²

.165

.443

.539

Adj. R²

.086

.354

.455

8.333***

10.220**ΔF✝, *, **, and *** denote significance at the 10%, 5%, 1%, and 0.1% levels, respectively(two-tailed). Estimated with ordinary least squares. Unstandardized coefficients are reported andstandard errors are in parentheses. VIF is less than 2.4. Results for medium and high degree of
DIVERSITY groups are not shown because regression lines for both groups are not statisticallysignificant (p> 0.10).

Simple slope analysis on financial performance for the low DIVERSITY group is
implemented when EU and BR take ±1SD, respectively. Figure 3 shows the results,

24which indicate that SPE enhances financial performance when EU is high and BR islow (β= 3.47, p= 0.000, β= 14.99, p= 0.001, respectively). These results are consistent
with those for the full sample analysis. Furthermore, R² of Models 3a and 4 is higher
than for the full sample analysis (R²= .651, .539, respectively). This result suggeststhat the exploratory power of changes of financial performance is higher than for thefull sample analysis. Hence, it can be said that SPE enhances financial performance
when EU is high, BR is low, and DIVERSITY is low13.

13

In order to confirm the robustness of these results, estimation not only for threegroups but also for two groups is implemented. Two groups are divided at the meanscore 4.7143. The results indicate that there are no prominent changes for performance
effects of the three-way-interaction term for either ROA or sales growth rate (β= −1.23,
p= 0.000, β= −5.75, p= 0.001, respectively). Furthermore, simple slope analysis shows
consistent results for the results of the three groups (β= 2.37, p= 0.001, β= 8.71, p=
0.006, respectively). Hence, the results of the supplementary analysis are not causedby grouping standards.

25

Figure 3
Effects of interaction between subjective performance evaluation, environmental
uncertainty, and budget rigidity on financial performance
when performance measure diversity is low

15
5

10

ROA

20

25

Panel A: ROA as a dependent variable

-4

-2

0
SPE

High EU, High BR
Low EU, High BR

Panel B: Sales growth rate as a dependent variable

5 Conclusion

26

2High EU, Low BR
Low EU, Low BR

4This study investigates the combined effects of subjective performance
evaluation, environmental uncertainty, and budget rigidity on financial performance.
The results based on a survey and archival data are consistent with expectations. In
other words, subjective performance evaluation enhances financial performance (both
ROA and sales growth rate) as environmental uncertainty increases and budget rigidity
decreases. Furthermore, ad hoc analysis indicates that positive performance effects ofsubjective performance evaluation become prominent when performance measure
diversity is low.
This study contributes to the growing body of related literature. First, this study
finds specific conditions in which subjective performance evaluation enhances
organizational performance. Previous research shows that contextual factors, such asenvironmental uncertainty, business unit strategy, and trust, determine theeffectiveness of subjective performance evaluation on organizational performance(Gibbs et al., 2004; Govindarajan, 1984; Govindarajan & Gupta, 1985). This study
finds that the effectiveness of subjective performance evaluation is determined not
only by congruence with environmental uncertainty, but also by congruence withbudget rigidity, which is not considered sufficiently by prior studies. In addition,
supplementary analysis suggests that the performance effects of subjective
performance evaluation under specific conditions become prominent when
performance measure diversity is low. Prior studies suggest there is a substitutional
relationship between subjective performance evaluation and performance measure
diversity (Hoppe & Moers, 2011; Luft et al., 2016). However, this study extends theline of research by revealing the effectiveness of the relationship by clarifying its
performance effects.
Second, this study finds effects of subjective performance evaluation on financial
performance. One subject of performance evaluation literature is whether outstanding
performance evaluation theoretically enhances financial performance (Franco-Santos
et al., 2012; Ittner et al., 2003). Although some literature examines the effects ofperformance measure diversity or alignment between business strategy andperformance measures on financial performance (Davis & Albright, 2004; Ittner et al.,2003), few studies examine the effects of subjective performance evaluation. The
results of this study contribute to this stream of literature by clarifying the financial
performance effects in certain situations.
Finally, the results of this study suggest the importance of empirically dividing
the two notions of performance evaluation style and budget rigidity. Research on
RAPM uses two terms, performance evaluation styles (objective, subjective, and both)and budget rigidity, without clear distinction (Govindarajan, 1984; Hartmann, 2000).

27

Some studies determine the cause of inconsistency of empirical results ortheory–practice gaps on undisposed notions (Hansen et al., 2003; Libby & Lindsay,
2010; Otley & Fakiolas, 2000). Against these problems that RAPM research embraces,
the results of this study in which two overlapping notions are clearly divided show that
financial performance effects vary depending on the combination of subjective
performance evaluation and budget rigidity in highly uncertain environments. It is
important to indicate the necessity of dividing the two overlapping notions by
empirical results because prior studies continue to suggest this theoretically.
The present study has some limitations. First, the performance effects ofcombining objective and subjective performance evaluation could not be considered in
the analysis. It can be assumed that performance evaluation based on both formal rules
and subjectivity of the evaluator’s perception exists in practice (Govindarajan, 1984;
Hopwood, 1972). The current study cannot clarify the optimal level of objectivity andsubjectivity in performance evaluation.
The second limitation relates to the problems of variable measurement ofsubjective performance evaluation and budget rigidity. It cannot be stated that thesurvey constructs of these notions are established. Furthermore, the measures ofbudget rigidity are diverse because there is no consensus about the definition of budget
rigidity. For this study, the statistical reliability of subjective performance evaluation
cannot be stated highly enough. Although the reliability of budget rigidity is sufficient,
there are some measurement errors, because of lack of consensus about its definition.
The third limitation is noise that financial performance effects contain. This
study measures financial performance as an aggregated variable at firm level, whereas
variables that use questionnaire items are measured as a core business unit. These
differences bring noise in financial performance effects (Van der Stede, Young, &Chen, 2005). Hence, it is appropriate to measure financial performance at business
unit level, not at firm level, in order to explain the performance effects of subjective
performance evaluation accurately,
The results of this study provide several avenues for future research. The first is
to examine the effects of the combination of subjective performance evaluation andbudget rigidity on organizational processes. Merchant (1990) cannot find mitigating
effects of the considerations of superiors that contain subjectivity on the manipulationof performance measures caused by high pressure of budgetary targets. In addition, theresults of this study indicate that performance effects vary depending on the
combination of subjective performance evaluation and budget rigidity. These results
provide the possibility that effects on organizational processes might vary in response
to combinations of subjective performance evaluation and budget rigidity. Specifically,28

it is important to examine managerial short-termism or manipulation of performancemeasures as dependent variables.
Furthermore, additional research is needed to examine the relationship between
subjective performance evaluation and performance measure diversity. The results ofthis study indicate that the performance effects of subjective performance evaluation
under specific conditions become prominent when performance measure diversity islow. Some studies indicate similar results (Hoppe & Moers, 2011; Luft et al., 2016),
although the reason for the results is not revealed sufficiently and future research is
required.
Finally, future research should examine the effects of subjective performance
evaluation on future organizational performance. Gibbs et al. (2004) find that
subjectivity in performance evaluation is positively associated with the extent oflong-term investments, such as training costs for employees. Furthermore, the resultsof this study show that subjective performance evaluation enhances the sales growthrate over multiple years in certain situations. Based on these results, it can bepresumed that subjective performance evaluation enhances future performance byincreasing investments for building organizational capabilities. Hence, future researchshould examine the performance effects of subjective performance evaluation not only
on short-term financial performance but also on long-term financial and non-financial
performance.

References

Aiken, L. S. and West, S. G. (1991). Multiple regression: Testing andinterpreting Interactions. London: Sage.
Arnold, M. C. and Artz, M. (2015). Target difficulty, target flexibility, andfirm performance: Evidence from business units’ targets.
Accounting, Organizations and Society, 40, 61-77.
Baker, G. P., Gibbons, R., and Murphy, K. J. (1994). Subjective
performance measures in optimal incentive contracts. Quarterly
Journal of Economics, 109(4), 1125-1156.

29

Bol, J. C. (2008). Subjectivity in compensation contracting. Journal of
Accounting Literature, 27, 1-24.
Bol, J. C. (2011). The determinants and performance effects of managers’
performance evaluation biases. The Accounting Review, 86(5),1549-1575.
Bol, J. C. and Smith, S. D. (2011). Spillover effects in subjective
performance evaluation: Bias and the asymmetric influence ofcontrollability. The Accounting Review, 86(5), 1549-1577.
Bruns, W. J. Jr. and Waterhouse, J. H. (1975). Budgetary control andorganizational structure. Journal of Accounting Research, 13,
177-203.
Cheng, M. M. and Coyte, R. (2014). The effects of incentive subjectivity
and strategy communication on knowledge-sharing and extra-role
behaviors. Management Accounting Research, 25, 119-130.
Davila, T. and Wouters, M. (2005). Managing budget emphasis throughthe explicit design of conditional budgetary slack. Accounting,
Organizations and Society, 30, 587-608.
Davis, S. and Albright, T. (2004). An investigation of the effect ofbalanced scorecard implementation on financial performance.
Management Accounting Research, 15(2), 135-153.
Dunk, A. S. (1993). The effect of budget emphasis and information
asymmetry on the relation between budgetary participation andslack. The Accounting Review, 68, 400-410.
Dunk, A. S. and Nouri, H. (1998). The determinants of budgetary slack: A
literature review and synthesis. Journal of Accounting Literature, 12,72-96.
Efron, B. and Tibshirani, R. J. (1986). Bootstrap methods for standard
errors, confidence intervals, and other measures of statisticalaccuracy. Statistical Science, 1(1), 54-77.
Efron, B. and Tibshirani, R. J. (1993). An introduction to the bootstrap.
London: Chapman & Hall.
Ekholm, B. and J. Wallin. (2011). The impact of uncertainty and strategy
on the perceived usefulness of fixed and flexible budgets. Journal ofBusiness Finance & Accounting, 38(1/2), 145-164.
Endo, K. (1994). Satei (Personal assessment) and interworker competitionin Japanese firms. Industrial Relations, 33(1), 70-82.
Feltham, G. and Xie, J. (1994). Performance measure congruity and30

diversity in multi-task principal/agent relations. The Accounting
Review, 69(3), 429-453.
Franco-Santos, M., Lucianetti, L., and Bourne, M. (2012). Contemporary
performance measurement systems: A review of their consequences
and a framework for research. Management Accounting Research,
23, 79-119.
Gibbs, M., Merchant, K., Van der Stede, W., and Vargus, M. (2004).
Determinants and effects of subjectivity in incentives. The
Accounting Review, 79(2), 409-436.
Govindarajan, V. (1984). Appropriateness of accounting data inperformance evaluation: An empirical examination ofenvironmental uncertainty as an intervening variable. Accounting,
Organizations and Society, 9(2), 125-135.
Govindarajan, V. and Gupta, A. K. (1985). Linking control systems tobusiness unit strategy: Impact on performance. Accounting,
Organizations, and Society, 10(1), 51-66.
Hair, J. F., Anderson, R. E., Tatham, R. L., and Black, W. C. (1998).
Multivariate data analysis, 5th edition. Upper Saddle River, New
Jersey: Prentice Hall.
Hansen, S. C., Otley, D. T., and Van der Stede, W. A. (2003). Practicedevelopments in budgeting: An overview and research perspective.
Journal of Management Accounting Research, 15, 95-116.
Hartmann, F. G. H. (2000). The appropriateness of RAPM: Toward thefurther development of theory. Accounting, Organizations andSociety, 25, 451-482.
Hemmer, T. (1996). On the design and choice of “modern” management
accounting measures. Journal of Management Accounting Research,
15, 51-74.
Holmstrom, B. (1979). Moral hazard and observability. Bell Journal of
Economics, 10(1), 74-91.
Holmstrom, B. and Milgrom, P. (1991). Multitask principal-agent analyses:
Incentive contracts, asset ownership, and job design. Journal of Law,
Economics, & Organization, 7, 24-52.
Hoppe, F. and Moers, F. (2011). The choice of different types ofsubjectivity in CEO annual bonus contracts. The Accounting Review,
86(6), 2023-2045.
Hopwood, A. G. (1972). An empirical study of the role of accounting data31

in performance evaluation. Journal of Accounting Research, 10,
156-182.
Hoque, Z. (2004). A contingency model of the association between
strategy, environmental uncertainty and performance measurement:
Impact on organizational performance. International Business
Review, 13, 485-502.
Hoque, Z. and James, W. (2000). Linking balanced scorecard measures tosize and market factors: Impact on organizational performance.
Journal of Management Accounting Research, 12, 1-17.
Ittner, C. D., Larcker, D. F., and Randall, T. (2003). Performance
implications of strategic performance measurement in financial
services firms. Accounting, Organizations and Society, 28, 715-741.
Libby, T. and Lindsay, R. M. (2010). Beyond budgeting or budgetingreconsidered? A survey of North-American budgeting practice.
Management Accounting Research, 21, 56-75.
Lillis, A. M. (2002). Managing multiple dimensions of manufacturingperformance: An exploratory study. Accounting, Organizations andSociety, 27, 497-529.
Luft, J., Shields, M. D., and Thomas, T. F. (2016). Additional informationin accounting reports: Effects on management decisions andsubjective performance evaluation under causal ambiguity.
Contemporary Accounting Research, 33(2), 526-550.
Marginson, D. W. and Ogden, S. (2005). Coping with ambiguity throughthe budget: The positive effects of budgetary targets on manager’s
budgeting behaviors. Accounting, Organizations and Society, 30,
435-456.
Merchant, K. A. (1985). Budgeting and the propensity to create budgetaryslack. Accounting, Organizations and Society, 10, 201-210.
Merchant, K. A. (1989). Rewarding results. Boston, Massachusetts:
Harvard Business School.
Merchant, K. A. (1990). The effects of financial control on data
manipulation and management myopia. Accounting, Organizations
and Society, 15(4), 297-313.
Merchant, K. A. and Manzoni, J. F. (1989). The achievability of budget
targets in profit centers: A field study. The Accounting Review, 3,
539-558.
Merchant, K. A. and Van der Stede, W. A. (2012). Management control

32systems: Performance measurement, evaluation and incentives, 3rdedition. London: Prentice Hall.
Nohria, N. and Gulati, R. (1996). Is slack good or bad for innovation?
Academy of Management Journal, 39(5), 1245-1264.
Otley, D. T. (1978). Budget use and managerial performance. Journal of
Accounting Research, 16, 122-149.
Otley, D. T. and Fakiolas, A. (2000). Reliance on accounting performancemeasures: Dead end or new beginning? Accounting, Organizations
and Society, 25, 497-510.
Ross, A. (1995). Job related tension, budget emphasis and uncertainty: A
research note. Management Accounting Research, 6, 1-11.
Sponem, S. and Lambert, C. (2016). Exploring differences in budget
characteristics, roles, and satisfaction: A configurational approach.
Management Accounting Research, 30, 47-61.
Todd, R. and Ramanathan, K. V. (1994). Perceived social needs, outcomesmeasurement and budgetary responsiveness in a not-for-profit
setting: Some empirical evidence. The Accounting Review, 69(1),122-137.
Van der Stede, W. A. (2000). The relationship between two consequencesof budgetary controls: Budgetary slack creation and managerial
short-term orientation. Accounting, Organizations and Society, 25,
609-622.
Van der Stede, W. A., Chow, C. W., and Lin, T. W. (2006). Strategy, choiceof performance measures, and performance. Behavioral Research in
Accounting, 18, 185-205.
Van der Stede, W. A., Young, S. M., and Chen, C. X. (2005). Assessing thequality of evidence in empirical management accounting research:
The case of survey studies. Accounting, Organizations and Society,30, 655-684.
Widener, S. K. (2007). An empirical analysis of the levers of control
framework. Accounting, Organizations and Society, 32, 757-788.
Woods, A. (2012). Subjective adjustments to objective performancemeasures: The influence of prior performance. Accounting,
Organizations and Society, 37, 403-425.
Wooldridge, J. M. (2013). Introductory econometrics: A modern approach,
5th edition. South-Western, OH: Cengage Learning.

33

APPENDIX A
Sample by Industries and Employees
Panel A: Sample by Industries (%)
Food

2.4

Fishery, Agriculture and Forestry

0.4

Textile Mill

3.6

Construction

6.0

Pulp and Paper

0.4

Electric Power and Gas

0.4

Chemical

8.7

Land Transportation

4.4

Drugs and Medicines

2.0

Marine Transportation.4

Oil and Coal

0.8

Warehousing and Harbor Transportation

2.0

Rubber2.0

Information Communication

6.0

Glass and Clay

2.0

Wholesale Trade

9.9

Steel

1.2

Retail Trade

11.9

Non-Ferrous and Fabricated Metal

1.6

Banks

4.8

Fabricated Metal

2.0

Securities and Commodities Futures.4

Machinery

4.8

Securities and Commodities Futures.4

Electrical and Electronics

8.3

Other Financing Business

1.2

Transportation Equipment

3.6

Real Estate

1.2

Precision Equipment

2.4

Services

5.2

Other Manufacturing

1.2

Panel B: Sample by Employees (%)
Less than 1,000

23.9%

1,000-2,499

25.9%

2,500-4,999

19.5%

5,000-9,999

13.2%

More than 10,000

17.5%

34

APPENDIX B
Descriptive Statistics of Questionnaire Items andResults of Principal Component Analysis
Mean

S. D.

Min

First-order

Second-orderloadingloadingMax

Subjective Performance Evaluation
Evaluate quantitative nonfinancial targets

4.02

1.32

1

7

.827

3.88

1.40

1

7

.708

4.80

1.33

1

7

.678

4.79

1.29

1

7

.634

subjectively on the basis of change ofconditions or explanation by manager
Evaluate budget target subjectively on thebasis of change of conditions or explanation
by manager
Evaluate manager’s abilities and attitudes
excepting objective performance
Evaluate qualitative nonfinancial targets
subjectively
Cronbach’s Alpha

0.68

Environmental Uncertainty
Diffusion of proprietary knowledge

3.56

.90

2

7

.750

−.227

Production, service, and information

3.84

.99

2

7

.729

−.315

Suppliers’ actions

3.90

.95

2

6

.720

.193

Customer demands, tastes and preferences

4.03

1.03

2

7

.719

−.343

Emergence of new competitors

4.37

1.30

1

7

.704

−.021

Market activities of competitors

4.13

1.02

2

7

.682

−.194

Industrial relations

3.21

1.03

1

6

.502

.642

Government regulation and policies

4.18

1.09

2

7

.519

.641

technologies

Cronbach’s Alpha

0.82

Budget Rigidity
Not achieving my budget has a strong impact

4.74

1.17

2

7

.842

4.44

1.17

1

7

.804

4.74

1.09

2

7

.751

on manager’s performance evaluation
Promotion prospects depend heavily on
ability to meet the budget
Achieving the budget is an accurate reflection

35

of whether managers are succeeding inbusiness
Performance of business unit’s managers are

5.46

1.16

1

7

.708

5.29

1.09

2

7

.699

4.89

1.18

1

7

.650

6.21

1.04

2

7

―

judged predominantly on the basis ofattaining budgetary targets
Control over business unit is achieved bymonitoring how well budget is on target
Not achieving the budget reflects poor
performance
Business unit’s managers constantly are
constantly conscious of meeting budgetary
targets
Cronbach’s Alpha

0.84

Performance Measure Diversity
Sales

5.54

1.28

2

7

Profits

6.22

.88

3

7

Operating cash flow

4.33

1.51

1

7

Rate of profits

5.56

1.15

2

7

Customer relevant

4.66

1.28

1

7

Internal-process relevant

4.72

1.24

1

7

Human development

4.54

1.21

1

7

Brand

4.27

1.25

1

7

Cronbach’s Alpha0.8036