Melco Management Accounting Research
Discussion Paper Series
No.MDP2019-007

Multidimensional Performance Evaluation Styles:
Budget Rigidity and Discretionary Adjustments

December 2019

Keita Masuya*
Faculty of Commerce, Takachiho University
19-1, Ohmiya 2chome, Suginami-ku, Tokyo, 168-8508 Japan
kmasuya@takachiho.ac.jp

Eisuke Yoshida
Faculty of Business and Commerce, Keio University*Corresponding author

The Melco Foundation
Nagoya, Japan
Discussion Paper Series of the Melco Management Accounting Research, in order to promote
the study of management accounting, has published this unfinished paper onto the Web. Pleaseobtain the permission of the author when citing this paper.

Multidimensional Performance Evaluation Styles:
Budget Rigidity and Discretionary Adjustments

Keita Masuya*
Faculty of Commerce, Takachiho University
19-1, Ohmiya 2chome, Suginami-ku, Tokyo, 168-8508 Japan
Tel: +81-(0)3-3317-4077
Fax: +81-(0)3-3313-9034
E-mail address: kmasuya@takachiho.ac.jp

Eisuke Yoshida
Faculty of Business and Commerce, Keio University
E-mail address: eyoshida@fbc.keio.ac.jp*Corresponding author

1

Multidimensional Performance Evaluation Styles:
Budget Rigidity and Discretionary Adjustments

Keita Masuya
Faculty of Commerce, Takachiho University, Tokyo, Japan

Eisuke Yoshida
Faculty of Business and Commerce, Keio University, Japan

Abstract
In management accounting research, there is no consensus on which stylesof performance evaluation lead to superior (inferior) performance. This
study addresses this issue by providing a new conceptual framework thatclassifies the styles of performance evaluation and revealing the styles thatlead to superior (inferior) performance. Based on a detailed literature
review, this study categorizes the styles of performance evaluation by twodimensions: the importance of budgetary targets and the manner of using
accounting information. Two concepts that explain these two dimensions,budget rigidity and discretionary adjustments, are specified. A congruent
combination of budget rigidity and discretionary adjustments is predictedin the case of environmental uncertainty. Empirical analysis is undertakenusing survey data from Japanese firms. The results indicate that a suitablecombination of budget rigidity and discretionary adjustments differs bysituation. As expected, a combination of lower budget rigidity and higher
discretionary adjustments is optimal in an uncertain environment.
Contrary to expectations, a combination of higher budget rigidity andhigher discretionary adjustments is optimal in a stable environment. This
study empirically demonstrates the importance of discretionary
adjustments for performance evaluation regardless of environmental
uncertainty.

2

Keywords
Budgets, Performance evaluation style, Multidimensionality, Budget rigidity,
Discretionary adjustments, Environmental uncertainty

1. Introduction
The relationship between styles of performance evaluation and their effects on
managers’ attitudes, behaviors, and performance is a central theme of management
accounting research (Luft and Shields, 2003). This area aims to reveal the situations inwhich managers’ dysfunctional behaviors arise. These studies assume that the use ofaccounting information by supervisors to evaluate managers’ performance causes such
behaviors. Since the seminal works of Hopwood (1972) and Otley (1978) produced
opposite results, several studies attempted to reveal the effects of performance
evaluation styles on managers’ attitudes, behaviors, and performance (Brownell, 1982;
Brownell and Hirst, 1986; Hartmann, 2000; Hirst, 1981) by specifying variables thatmoderate this relationship (Brownell and Hirst, 1986; Hartmann, 2000; Hirst, 1983).
In addition, many studies focused on measuring variables and analytical techniques(Brownell, 1982, 1985; Derfuss, 2009; Otley and Fakiolas, 2000). Nevertheless, there
is no consensus on which performance evaluation style leads to superior (or inferior)performance.
One serious problem of prior studies is that they do not reconsider concept of
performance evaluation styles theoretically. They treat the concept is treated vaguely,
which hinders the accrual of systematic knowledge. Performance evaluation style is aconstructive concept with several dimensions (Briers and Hirst, 1990; Hartmann et al.,2010; Hopwood, 1972). Hence, it is necessary to clarify the dimensions of
performance evaluation styles. Nevertheless, several studies focused on only one
dimension, because they do not recognize their multidimensionality. In addition, manystudies label the concept of performance evaluation styles the same way, even though
the meaning differs across studies.
This study aims to investigate the performance evaluation styles that lead tosuperior (inferior) performance. Hence, this study first seeks to clarify the concept of
performance evaluation styles, specifically by assuming that they consist of two
dimensions: the priority of budgetary targets among the performance criteria and how
supervisors use accounting information in managerial performance evaluation. The
first dimension explains the factor measured as a performance criterion at the stage ofsetting performance criteria and the second dimension indicates whether supervisors3

use accounting information strictly or leniently ex post. In addition, this study assumes
that two concepts underpin these two dimensions: budget rigidity and discretionary
adjustments.
Then, this study assumes that congruence between the degree of budget rigidityand discretionary adjustments is determined by contextual variables. In particular, thepresent study focuses on environmental uncertainty, which is one of the mostimportant issues of interest to researchers (Hartmann, 2000; Merchant, 1990; Ross,
1995). This study assumes that environmental uncertainty determines congruence
between the degree of budget rigidity and discretionary adjustments.
This relationship is tested empirically using survey data collected from Japanesefirms. The results indicate that a suitable combination of budget rigidity anddiscretionary adjustments differs according to environmental uncertainty. Specifically,
the combination of lower budget rigidity and higher discretionary adjustments
enhances organizational performance under a highly uncertain environment. This
study also posits that a combination of higher budget rigidity and lower discretionary
adjustments is suitable in a stable environment, though the results show that a
combination of higher budget rigidity and higher discretionary adjustments improvesorganizational performance in a stable environment. These results indicate that theactive use of discretionary adjustments complements the effects of budget rigidity onorganizational performance, regardless of environmental uncertainty.
This study offers several contributions to the literature on performance
evaluation. First, it provides a new conceptual framework that classifies performance
evaluation styles. While performance evaluation styles are a composite concept withseveral dimensions, prior studies treated these dimensions vaguely (Briers and Hirst,
1990; Hartmann et al., 2010). The present study addresses this problem byreconsidering the concepts and specifies two dimensions for performance evaluation.
Moreover, this study specifies two concepts, budget rigidity and discretionary
adjustments, underpinning the two dimensions. This reconsideration of the theory isimportant for providing guidelines for the development of research on performance
evaluation styles.
Second, this study enhances the literature on effective (ineffective) performance
evaluation. The results of the empirical analysis suggest that it is difficult tounderstand the performance implications of performance evaluation styles without
recognizing their multidimensionality. In addition, the appropriate combination ofbudget rigidity and discretionary adjustments differs according to environmental
uncertainty. These results are important because most prior studies examined the4contingency relationship between only one dimension of performance evaluation andspecific contextual variables (Brownell, 1982; Hartmann, 2000; Hirst, 1983).
Third, the results indicate the importance of discretionary adjustments inbudget-based performance evaluations. Specifically, discretionary adjustmentscomplement the effects of budget rigidity, regardless of environmental uncertainty.
Prior studies suggest that whether managers adopt dysfunctional behaviors depends
not on budget-based performance evaluation per se, but on how supervisors use
accounting information to evaluate managerial performance (Derfuss, 2009; Hopwood,
1972; Sponem and Lambert, 2016). Moreover, some studies demonstrated that
including subjectivity in budget-based performance evaluations is an effective way to
improve future performance (Aranda et al., 2019). Considering the results of this andprior studies, discretionary adjustments complement the motivational effects ofperformance targets. From this perspective, the results demonstrate that the use of
discretionary adjustments to evaluate budget-based performance seems to be rational
in practice.
This study has several important implications for managerial practice. The
results show that practitioners should recognize the importance of budgetary targets
and the use using accounting information in performance evaluation as two
performance evaluation dimensions. Supervisors must address both dimensions toenhance desirable behaviors and improve performance. In addition, the use of
discretionary adjustments in performance evaluation is important for organizational
success. Supervisors should not evaluate managers’ performance mechanically andsystematically based on achieving targets. Instead, supervisors should consider why
managers cannot achieve targets, how they behave in the budgeting period, and theiractions, among other factors. Such evaluation efforts can direct managers to adopt
behaviors desirable for the organization, which ultimately leads to organizational
success.
The remainder of this paper proceeds as follows. Section 2 reviews the literature
on performance evaluation styles and introduces the new conceptual framework thatclassifies the performance evaluation styles on two dimensions and explains them withtwo concepts: budget rigidity and discretionary adjustments. Section 3 develops theresearch hypotheses by building on complementary theory. Section 4 explains the datacollection and variable measurements. Section 5 presents the results of the empiricalanalysis. Finally, Section 6 summarizes the study, its limitations, and directions for
future research.

5

2. Theory
2.1

Prior studies
The relationship between budget-based performance evaluation and managers’

behaviors first received attention about 60 years ago. Descriptive field studies focusedon how factory managers or budget staff use budgetary control systems, andhighlighted the interpersonal problems caused by their use (Argyris, 1952; Hofstede,1967). These studies described serious problems arising from the use of budgets aspressure devices, such as hostility toward top management or deteriorating relations
among supervisors, subordinates, and colleagues.
Hopwood (1972) advanced the initial studies dramatically by classifying
performance evaluation styles and testing their effects empirically. Specifically,
Hopwood (1972) proposed three performance evaluation styles—budget-constrained,
profit-conscious,andnon-accounting.

Hopwood(1972)

found

that

a

budget-constrained style, which refers to the strict use of budgetary information toevaluate performance, causes the most serious problems.
However, subsequent studies do not replicate Hopwood’s (1972) results. Otley’s(1978) study was the first to produce contrary results, and numerous studies since thenattempted to understand why (Brownell, 1982; Brownell and Hirst, 1986; Hartmann,
2000; Hirst, 1981). Studies following Hopwood (1972) and Otley (1978) developed in
two directions. The first is toward the contingency relationship, or the conditions
under which performance evaluation styles cause (reduce) dysfunctional behaviors(Brownell and Hirst, 1986; Hirst, 1981, 1983) with contextual variables including
environmental uncertainty, task complexity, and budgetary participation. The otherdirection addresses variable measurement and analytical techniques (Brownell, 1982,
1985; Derfuss, 2009; Otley and Fakiolas, 2000). Nevertheless, there have been no
advances in understanding the situations in which managers’ dysfunctional behaviors
are enhanced (reduced).
2.2

Multidimensional nature of performance evaluation styles
Despite the substantial body of work, few studies reconsidered the conceptual

framework or concepts of the styles of performance evaluation. In particular, priorstudies did not sufficiently recognize the multidimensional nature of performance
evaluation styles. This is a serious problem, as prior studies focused on only one
dimension or labeled multiple dimensions in the same way, even though these studies

6

apply different definitions. These problems inhibit the accrual of systematic
knowledge on the performance effects of performance evaluation styles.
A few studies offered a theoretical consideration of the concept of performance
evaluation styles. Briers and Hirst (1990) suggested that they consist of at least three
dimensions: the range of criteria used to evaluate performance, the flexibility withwhich variances from the standard are interpreted, and the manner in which managers
handle short- and long-term concerns. Similarly, Hartmann et al. (2010) posit two
dimensions. The first is the measurement dimension to explain the range ofperformance criteria for performance evaluation. The range includes quantitative andqualitative performance measures. The second is the style dimension, which explains
the manner (e.g., rigid or lenient) in which supervisors use these performancemeasures ex-post. Hartmann et al. (2010) invoked the concept of leadership style toexplain this dimension. Using these two concepts, they examine empirically the effects
on managers’ work-related attitudes.
Hartmann et al.’s (2010) classification has two problems. First, it cannot explain
the strict use of accounting information. Specifically, they classified the styledimension by whether a performance measure is quantitative or qualitative.
Quantitative performance measures include both financial and non-financial measures.
Their classification cannot recognize the differences between these two attributes; thatis, their classification cannot explain the use of “accounting” information strictly. The
second problem is that their concept of leadership style, such as initiating structure,cannot explain the manner in which managers use accounting information strictly.
Specifically, the concept of leadership style is too general to explain how supervisors
use accounting information. Therefore, there is scope to make the concepts moresophisticated.
2.3

Conceptual framework
Based on the abovementioned studies, the authors assume that the concept of

performance evaluation styles consists of two dimensions. The first is the priority ofbudgetary targets, which explain the positioning of budgetary targets as evaluation
criteria. Specifically, this dimension focuses on whether budgetary targets are of highor low priority at the stage of setting performance criteria. Prior studies discussed thisdimension as a range of measures to evaluate performance (Briers and Hirst, 1990;
Hartmann et al., 2010).
The other dimension is how supervisors use accounting information. Thisdimension explains whether supervisors evaluate managers’ performance strictly or

7

flexibly based on performance measures ex post. In other words, this dimension
explains the degree of subjective adjustments at the ex-post stage of performance
evaluation. Previous studies discussed this dimension as the flexibility with which
supervisors interpret variances from the standard or how they handle short- andlong-term concerns (Briers and Hirst, 1990; Hartmann et al., 2010; Hopwood, 1972).
The distinction between these two dimensions is important because no
significant understanding of the effects of performance evaluation styles is possible
without considering both. Recognizing performance evaluation styles by only one
dimension leads to several problems. Focusing only on the priority of budgetary
targets ignores how supervisors use accounting information ex post. One can assume
that managers are less likely to adopt dysfunctional behaviors to achieve budgetary
targets if supervisors evaluate managers’ performance flexibly ex post. Hence,
ignoring this dimension erodes the understanding of the effects of performance
evaluation styles.
Similarly, focusing on only how supervisors use accounting information ex post(the use dimension) ignores the priority of budgetary targets. Performance measuresand targets themselves direct managers’ attention toward them, regardless of whether
supervisors use accounting information strictly or leniently for ex-post performance
evaluation (Flamholtz, 1983; Marginson and Ogden, 2005; Sprinkle, 2003).
Attention-directing effects become significant as the priority of budgetary targets
increases, and thus, focusing on only the use dimension ignores the serious effects ofbudgetary targets. In this way, the relationship of the two dimensions of performance
evaluation styles is not unidirectional but reciprocal.
It is important to specify the concepts suited to explain these two dimensions.
Otherwise, this theoretical reconsideration would not produce an empirical analysis.
This study posits several concepts that explain these two dimensions. Budget rigidity
explains the priority of budgetary targets. Budget rigidity increases if supervisorsemphasize budgetary targets as comprehensive measures that reflect managers’ efforts
and behaviors. On the contrary, budget rigidity decreases as the priority of
performance targets besides budgetary targets increases.
Discretionary adjustments are the concept that explains the use dimension. This
concept explains the degree to which supervisors evaluate managers’ performance
subjectively ex post. Prior studies showed the effectiveness of discretionary
adjustments as a means of improving goal congruence and risk reduction (Bol, 2008;
Gibbs et al., 2004; Hoppe and Moers, 2011). Research on budgeting also demonstrated
the wide diffusion of discretionary adjustments in budget-based performance8

evaluation (Libby and Lindsay, 2010). Discretionary adjustments increase as
supervisors evaluate managers’ performance flexibly in consideration of theenvironment or managers’ explanations. On the contrary, discretionary adjustments
decrease as supervisors evaluate managers’ performance based on ex-ante
performance criteria strictly.
As Fig. 1 shows, performance evaluation styles are classified by combining thedegree of budget rigidity and discretionary adjustments, thus yielding four
performance evaluation styles. First, the non-accounting style places a lower priority
on budgetary targets and supervisors hardly judge subordinates’ performance
subjectively. Supervisors set performance targets based on quantitative non-financial
measures and evaluate them. Second, the flexible evaluation style is a performance
evaluation that does not emphasize budgetary targets and supervisors often judge
subordinates’ performance subjectively. In this style, supervisors emphasize
quantitative and/or qualitative non-financial performance targets and evaluate theachievement of budgetary targets flexibly. Third, the budget emphasis style makes
budgetary targets the primary priority and supervisors evaluate managers’ target
achievements objectively. Achievement of budgetary targets is primary. Finally, thebudget emphasis and flexible evaluation style, requires that subordinates achievebudgetary targets, but with a subjective performance assessment.
Low

High

Low

Non-accounting styleFlexible styleHigh

Budget emphasis style

Budgetrigidity

Budget emphasis andflexible style

Fig. 1

Classification of styles of performance evaluation

3. Hypothesis
Many prior studies in this area adopted contingency theory (Hartmann, 2000),
which is useful to examine control practices that match contextual variables, but islimited in that it cannot explain interdependence (internal consistency) among control
practices (Grabner and Moers, 2013). This study adopts complementary theory to
examine the interdependence of budget rigidity and discretionary adjustments whileconsidering the effects of a contextual variable (environmental uncertainty). This
theory is useful for predicting how each dimension of performance evaluation style fits
together (Grabner and Moers, 2013). In other words, it is suitable for explaining and9examining the interdependence of budget rigidity and discretionary adjustments inspecific contexts (uncertain vs. stable external environments).
3.1

Uncertain environments
In an uncertain environment, random noise leads to fluctuations in theachievement of budgetary targets. Financial performance measures, such as budgetary
targets, cannot give supervisors accurate information about managers’ efforts (Banker
and Datar, 1989; Hirst, 1981, 1983; Holmstrom, 1979; Moers, 2006). In this case,prioritizing budgetary targets enhances the risk that managers will have lower
performance (Banker and Datar, 1989; Hirst, 1981; Holmstrom, 1979). In addition,
managers perceive ambiguity about the causal relationship between their own efforts
and consequences. Managers who cannot predict how to achieve targets perceive ahigher cost of making efforts, which provides less incentive to achieve targets.
Nevertheless, emphasizing budgetary targets raises the risks that managers will
legitimize dysfunctional behaviors such as manipulating performance measures(Hopwood, 1972; Merchant, 1990). Such behaviors are problematic because managers’
make efforts in the wrong direction. Hence, in a highly uncertain environment, it issuitable to make budgetary targets a lower-priority performance criterion (Holmstrom,1979).
In addition, it is possible to predict the benefits of lowering budget rigidity.
Specifically, it not only reduces the risks that managers bear, but also gives them
opportunities to adapt their behaviors to environmental change. Short-term
performance pressure decreases as budget rigidity decreases. Consequently, managers’
long-term orientation is enhanced (Merchant, 1990; Van der Stede, 2000). In addition,
they quickly make adjustments for customer needs and engage in innovative behaviors(Lillis, 2002; Van der Stede, 2000). It can be assumed that lower budget rigidity issuitable in a highly uncertain environment because these orientations and behaviors
are more important in this case.
Here, it should be noted that lower budget rigidity is not enough to motivate
managers to adapt their behaviors to environmental change. Identical performance
evaluation requires not only a reduction in managers’ risks, but also a motivation forthem to adapt their behaviors to environmental change (Gibbs et al., 2004).
Discretionary adjustments enable supervisors to use ex-post information to judge
managers’ efforts and behaviors flexibly (Bol, 2008; Gibbs et al., 2004; Hoppe andMoers, 2011), which allows managers to predict how new information, such as
fluctuations in the external environment, changes supervisors’ priorities. Managers

10will act autonomously based on their predictions (Bol, 2008). Therefore, discretionary
adjustments encourage managers to adapt their behaviors to changes in the external
environment (Bol, 2008; Gibbs et al., 2004).
Based on this discussion, this study predicts that a combination of lower budget
rigidity and higher discretionary adjustments is suitable in a highly uncertain
environment. This combination can reduce the risk that managers’ performance will
decline. Lower budget rigidity and higher discretionary adjustments reinforce eachother, leading to risk reduction and goal congruence. Therefore,
Hypothesis 1: In an uncertain environment, discretionary adjustments moderate
the relationship between budget rigidity and performance. The positive effects oflower budget rigidity on performance become significant as discretionary adjustments
increase.
3.2

Stable environments
In a stable environment, financial performance measures such as budgetary

targets, can are comprehensive measures that reflect managers’ behaviors and efforts
accurately. In this situation, the higher priority of budgetary targets enhances goal
congruence between supervisors and managers. Managers can explicitly understandthe causal relationships between their efforts and consequences. Hence, higher budget
rigidity motivates managers to achieve budgetary targets (Hirst, 1981; Moers, 2006;
Otley, 1978). Furthermore, achieving budgetary targets leads to organizational success.
Regarding the use dimension, active use of discretionary adjustments is notdesirable. In a stable environment, it is easier to clarify managers’ optimal behavior. In
addition, it is not difficult to design desirable formal evaluation rules ex ante to givesupervisors information that accurately reflects managers’ efforts. Managers can easily
estimate the return on their own efforts when supervisors evaluate their performancebased on formal rules (Baker et al., 1994; Bol, 2008). This estimation motivates
managers to achieve budgetary targets. Prior studies also indicated that objective
performance evaluation is suitable when performance measures reflect managers’
efforts accurately (Baker et al., 1994; Govindarajan, 1984).
From this discussion, the authors predict that the combination of higher budget
rigidity and lower discretionary adjustments is optimal in a stable environment. In
other words, managers face fewer risks and goal congruence between supervisors andmanagers increases as budget rigidity rises and discretionary adjustments decrease.
This combination creates the maximum motivation for managers to achieve budgetary11

targets. Hence,
Hypothesis 2: In a stable environment, discretionary adjustments moderate therelationship between budget rigidity and performance. The positive effects of higher
budget rigidity on performance become significant as discretionary adjustments
decrease.
Fig. 2 shows the optimal combinations this study assumes. The cell labeled“Unfit” refers to an ineffective style of performance evaluation. These practices do notsuit the situation. In other words, these firms do not attain equilibrium. Therefore, it
can be assumed that the performance of these firms is lower than that of other firms
whose practices suit the situation.
Discretionary adjustments
low

high
congruent under uncertain

low

unfit

environment(hypothesis 1)

Budgetrigidity

congruent underhighstable environment

unfit(hypothesis 2)

Fig.2

Assumed congruent relationships

4. Methods
4.1

Data collection
To test the hypotheses, in 2014, questionnaires were sent to 1,822 firms listed on

the First Section of the Tokyo Stock Exchange. The respondents of the mail survey are
managers/directors of corporate planning departments, who are appropriate
respondents because their main functions in large Japanese firms include exploring
appropriate budgeting and performance evaluation systems. To increase the response
rate, a once-off follow-up postcard was sent to the managers/directors who did not
respond by the due date. These processes resulted in 308 responses (a 16.9% responserate). There were 233 final samples for the analysis after eliminating firms that did notuse budgets (2), did not use budgetary targets to evaluate the performance of business
unit managers (32), had missing data (22), and for which archival data onorganizational performance was unavailable (19).

12

4.2
4.2.1

Variables
Budget rigidity
Budget rigidity (BR) is the priority of budgetary targets among the evaluation

criteria. Prior studies often used instruments developed by Hopwood (1972) or Otley(1978) to measure BR. However, they may not be appropriate because they weredeveloped to investigate practices at particular research sites (Otley and Fakiolas,2000). In this study, BR is measured following Van der Stede’s (2000) seven items
developed for a cross-sectional survey. The seven items are measured on Likert scalesof 1 to 7, where 1 indicated “not at all” and 7 “absolutely correct.” One item waseliminated due to the ceiling effect.
4.2.2

Discretionary adjustments
Discretionary adjustments (DA) occur when supervisors evaluate managers’

performance flexibly by considering their own judgements and opinions. DA ismeasured as follows. First, items that measure discretionary adjustments based onbudgetary targets are adopted from Libby and Lindsay’s (2010) questionnaire. Second,
respondents are asked to rate objective non-financial measures subjectivity. These twoitems are measured on Likert scales of 1 to 7, where 1 indicated “not at all” and 7“absolutely correct.”
4.2.3

Environmental uncertainty
Environmental uncertainty (EU) is measured by questionnaire items rather thanobjective archival data. Specifically, EU is measured by eight items that reflect threecommon dimensions in prior studies: internal business processes, competition, andtechnology (Ekholm and Wallin, 2011). These eight items are measured on Likert
scales of 1 to 7, where 1 indicated “very predictable” and 7 “very unpredictable.”
Table 1 shows the results of the exploratory factor analysis using allquestionnaire items via maximum likelihood extraction and oblique rotation. The
loadings of the scale items for each factor vary from .482 to .797 for BR, .609 to .999
for DA, and .507 to .668 for EU. The Cronbach’s alphas are .838, .756, and .830 for
BR, DA, and EU, respectively. These results support the reliability of the measurement
items. In addition, Table 1 supports the convergent and discriminant validity of themeasurement items. This study does not use two questionnaire items that do not load
on factors for the variable measurement and empirical analysis.

13

Table 1 Descriptive statistics and the results of exploratory factor analysis
Factor loading
Questionnaire item

Mean

Not meeting the budget has a strong impact on managers’

S. D.

Min

Max
BR

DA

EU

4.76

1.169

2

7

.797

.074

.287

Promotion prospects depend heavily on ability to meet the budget

4.48

1.153

1

7

.692

.140

.271

Meeting the budget is an accurate reflection of whether managers

4.76

1.100

2

7

.611

.124

.260

5.46

1.159

1

7

.590

-.014

.250

5.29

1.107

2

7

.588

.045

.232

Not meeting the budget reflects poor performance

4.90

1.188

1

7

.482

-.018

.296

Budget target is evaluated subjectively based on change of

3.96

1.386

1

7

-.001

.999

.001

4.07

1.313

1

7

.017

.609

.037

Production, service, and information technologies

4.17

1.002

1

6

-.260

-.195

.668

Diffusion of proprietary knowledge

4.44

.899

1

6

-.412

-.137

.625

Emergence of new competitors

3.61

1.302

1

7

-.184

-.092

.622

Customer demand, tastes, and preferences

3.97

1.046

1

6

-.280

-.155

.608

Suppliers’ actions

4.12

.947

2

6

-.318

-.047

.547

Market activities of competitors

3.87

1.042

1

6

-.329

-.137

.507

Industrial relations

4.82

1.033

2

7

-.184

-.031

.329

Government regulation and policies

3.83

1.109

1

6

-.254

-.029

.296

.838

.756

.830

performance evaluation

are successful
Performance of business unit’s managers is assessed predominantly
on the basis of attaining budgetary targets
Control over business unit is achieved by monitoring how well thebudget targets are on track

conditions or explanation by manager
Quantitative non-financial targets are evaluated subjectively basedon change of conditions or explanation by manager

Cronbach’s α
BR = budget rigidity, DA = discretionary adjustments, EU = environmental uncertainty.

For ease of presentation, loading > .400, which are used in the final measurement of constructs, are highlighted in bold.

4.2.4

Organizational performance
Organizational performance is measured by financial and market performance inline with prior studies (Ittner et al., 2003; Said et al., 2003) in order to determine theeffects of performance evaluation styles on multiple facets of performance. First,
financial performance is measured by the average ROA of the 3 years between 2011
and 2014. Second, the market performance is the Tobin’s Q in 2014, which indicates14the market value of a company divided by the replacement cost of its assets. To control
for industry effects on organizational performance, both ROA and Tobin’s Q are
divided by industry average scores.
4.2.5

Control variables
To observe the cleanest possible effects of performance evaluation styles, theanalysis includes several firm characteristics. The first is organizational size (SIZE),
which is an important factor influencing the use of budgetary control systems and theireffectsonperformance(BrunsandWaterhouse,

1975;

Merchant,

1981).

Organizational size is measured by the natural logarithm of total assets.
Second, debt leverage (Leverage) and the ratio of free cash flow to total assets(FCF) are included. Prior studies indicated that the financial condition of the firm
significantly influences its use of budgetary control systems to evaluate performance(Becker et al., 2016).

5. Results
5.1

Descriptive statistics and variable correlation
Table 2 presents the descriptive statistics and variable correlations. Panel B

shows that the correlation coefficient of the two dimensions of performance evaluationstyles (BR and DA) and organizational performance (ROA and Tobin’s Q) are notstatistically significant (p> .10), suggesting that it is not possible to understand theperformance effects of performance evaluation styles without considering themultidimensionality of performance evaluation styles.
5.2

Hypothesis tests
The hypotheses were tested using a multiple regression analysis. This method issuitable for examining the interrelationships among control practices (Bedford andMalmi, 2015; Grabner and Moers, 2013). This study also investigates the interaction
effects of BR and DA, which comprise performance evaluation styles. Hence, multiple
regression analysis is an appropriate analytical technique.
Three models are estimated to examine the hypotheses. Model 1 contains eachmain exploratory variable (BR, DA, and EU) and includes the control variables. Twoand three-way interaction terms are included in Models 2 and 3, respectively. It should
be noted that the interaction terms produce a multicollinearity problem, which we
avoid by centralizing all independent variables. Subsequently, the interaction terms areformed.

15

Table 2 Descriptive statistics and variable correlation
Mean

S. D.

Min

Max

Panel A Descriptive statistics
1

BR

4.94

.85

2.00

7.00

2

DA

4.02

1.21

1.00

7.00

3

EU

3.97

.77

1.83

6.00

4

ROA

-.43

4.44

5

Tobin’s Q

-.03

.29

6

SIZE

7.96

1.42

7

LEVERAGE

49.80

17.82

8

FCF

1.83

6.29

1

2

3

4

5

6

7

Panel B Variable correlation
1

BR

2

DA

.082

3

EU

-.052

-.138**

4

ROA

-.064

.056

.086

5

Tobin’s Q

-.093

.068

.099

.647***

6

SIZE

.086

-.006

-.070

.086

.059

7

LEVERAGE

-.053

.040

-.126*-.368***.041.248***

8

FCF

.047

.012

.145**

.079

.048

.028

-.205***

Pearson correlations. *, **, and *** denote significance at the 10%, 5%, and 1% levels, respectively(two-tailed). A blank space is left where we can specify the responding firms.
Variable definition
BR = budget rigidity, DA = discretionary adjustments, EU = environmental uncertainty, ROA = return on
assets, SIZE = organizational size, LEVERAGE = debt leverage, FCF = ratio of free cash flow to total assets

Table 3 reports the multiple regression analysis regression results. Initially, themain effects of BR on ROA are statistically significant, but only at .10 in Model 1 (β=−.479, p= .076). The main effects of DA on ROA and Tobin’s Q are not statisticallysignificant (p> .10). Prior studies on DA concur that DA is not always effective
because it comes with both benefits and costs (Baker et al., 1994; Bol, 2008; Gibbs etal., 2004). The results showing that the main effects of DA are not statisticallysignificant are consistent with prior studies.
Model 2 shows negative coefficients of BR*EU on ROA and Tobin’s Q (β=16Table 3 Effects on organizational performance of association between budget rigidity,
discretionary adjustments, and environmental uncertainty
Dependent

SIZE

LEVERAGE

FCF

BR

DA

EU

ROA

Tobin’s Q

Model 1aModel 2aModel 3aModel 1bModel 2bModel 3b.642***.613***.596***

.013

.012

.010

(.195)(.191)(.189)(.014)(.013)(.013)-.106***-.107***-.099***

.001

.001

.001

(.016)(.016)(.016)(.001)(.001)(.001)

-.013

-.042

-.055

.002

.001

.000

(.044)(.044)(.044)(.003)(.003)(.000)-.479*-.455*-.489*

-.029

-.029

-.032*(.269)(.263)(.261)(.019)(.019)(.018)

.405

.391

.263

.026

.025

.014

(.270)(.264)(.266)(.019)(.019)(.019)

.253

.245

.245

.032

.029

.029

(.273)(.268)(.265)(.019)(.019)(.019)

.157

.123

-.017

-.020

(.251)(.248)(.018)(.017)-.970***-.930***-.075***-.071***(.272)(.269)(.019)(.019)

.106

-.051

.014

-.001

(.231)(.237)(.016)(.017)

BR*DA

BR*EU

DA*EU

BR*DA*EU

Constant-.543**-.050***(.219)(.015)

-.235

.031

-.204

-.174

-.162

-.184*(1.572)(1.542)(1.527)(.112)(.109)(.107)

Δ R²

,047

.021

.069

.041

ΔF

4.578***

6.137**

5.755***

10.580***

R²

.190

.237

.257

.034

.103

.144

Adjusted. R²

.168

.206

.224

.008

.067

.106

8.815***

7.681***

7.686***

1.320

2.854***

3.737***

F*, **, and *** denote significance at the 10%, 5%, and 1% levels, respectively (two-tailed). Estimations
are with ordinary least squares. We report unstandardized coefficients and standard errors in parentheses.
The variance inflation factor is less than 1.5. See Table 2 for definitions of the variables.

17

−.970, p= .000, β= −.075, p= .000, respectively). These results reflect theincongruence between higher BR and higher EU. That is, it is suitable to lower thepriority of budgetary targets as a performance criterion in a highly uncertain
environment.
Finally, Model 3 shows statistically significant coefficients on BR, DA, and EU,
using either ROA or Tobin’s Q (β= −.543, p= .014, β= −.050, p= .001). Moreover, theincrease in R² is statistically significant (ΔR²= .021, ΔF = 6.137, p= .014, ΔR²= .041,
ΔF = 10.580, p= .001, respectively). These results indicate that the effects of BR on
organizational performance depend on both DA and EU.
Next, a simple slope analysis is performed in each EU group to understand thecontent of the interaction. Following Aiken and West (1991), the regression line of BR
on organizational performance when DA and EU take ±1SD is estimated.
Fig. 3 shows the patterns of the BR regression line on organizational
performance.
Regarding Hypothesis 1, the results indicate that BR decreases both ROA and
Tobin’s Q when both DA and EU are higher (β= −2.446, p= .000, β= −.197, p= .000).
As in Fig. 3, organizational performance increases as BR decreases when both DA and
EU increase. In other words, the positive effects of a lower BR on organizational
performance are reinforced as both DA and EU increase. Thus, Hypothesis 1 is
supported. These results can be interpreted as follows. It is suitable to reduce thepriority of budgetary targets and to evaluate managers’ performance flexibly whileconsidering managers’ behaviors and efforts in a highly uncertain environment.
Next, regarding Hypothesis 2, the statistical significance of the simple slope of
BR on organizational performance cannot be confirmed when both DA and EU
decrease (p> .10). Nevertheless, the results indicate that BR enhances both ROA and
Tobin’s Q when DA is higher and EU is lower (β= 1.669, p= .005, β= .079, p= .038).
These results indicate that organizational performance is enhanced as both BR and DA
increase in a stable environment. Hence, the results do not support Hypothesis 2. The
results for Hypothesis 2 suggest that the active use of DA reinforces the positiveeffects of BR on organizational performance in a stable environment.
These results can be interpreted from the strength and direction of motivation.
From the perspective of motivation strength, Aranda et al. (2019) showed empiricallythat subjective performance evaluation enables managers to set stretch targets andmotivates managers to make efforts to achieve targets. Thus, future performance isenhanced. Considering their results, it is possible that the active use of discretionary
adjustments enables firms to set difficult budgetary targets and motivates managers to

18

1

ROA

0

-1

-2

Low BR

High BR

Low BR

High BR

Tobin’s Q

.1

0

-.1

Fig. 3 Results of simple slope analysis
achieve targets, even when the external environment is stable. In other words, higher
discretionary adjustments complement the motivational effects of budget rigidity.
Therefore, it can be assumed that the combination of high budget rigidity and highdiscretionary adjustments is congruent in a stable environment.
In terms of motivation direction, the results suggest the importance of
discretionary adjustments as a means of managing the process of achieving budgetary
targets. Specifically, a low degree of discretionary adjustments means that supervisors
evaluate managers’ performance mechanically or systematically based on whether theyreach budgetary targets. This type of performance evaluation only emphasizes
performance outcomes and does not consider the process of achieving targets. It raises

19the risk that managers adopt dysfunctional behaviors such as manipulating
performance measures that inhibit business improvement. Therefore, the active use of
discretionary adjustments is possibly effective for managing the processes ofachieving targets, even when the external environment is stable.
The results also suggest the importance of discretionary adjustments. That is,
discretionary adjustments complement the effects of budget rigidity on organizational
performance regardless of environmental uncertainty. Prior studies noted that whether
managers adopt dysfunctional behaviors depends not on budget-based performance
evaluations per se, but on the manner in which supervisors use accounting information
to evaluate performance (Derfuss, 2009; Hopwood, 1972; Sponem and Lambert, 2016).
Furthermore, previous studies showed that performance evaluation practices that
include subjectivity are widespread (Libby and Lindsay, 2010). The results here
indicate the rationality of such practices; specifically, that it is important to notevaluate managers’ performance based on accounting information mechanically orsystematically, but rather to evaluate performance flexibly while considering managers’
actual efforts to achieve targets. In this sense, the budget emphasis and flexible
evaluation style can be referred to as a reciprocal style in a stable environment. This
style requires subordinates to achieve budgetary targets that embody organizational
goals while adjusting their performance subjectively. Both organizational goals and thecontext that subordinates face are considered. Hence, this style develops a reciprocal
relationship between supervisors and subordinates in a stable environment.
Fig. 4 shows the congruent combinations of the empirical analysis.

Discretionary adjustments
Low

High
Congruent under uncertain

Low

Unfit
environment

Budgetrigidity

Congruent under
High

Unfit
stable environment

Fig. 4

Empirically supported congruent relationships

20

6. Conclusion
This study aimed to determine the performance evaluation styles that lead tosuperior (inferior) performance. The many studies in this area did not reach aconsensus on this relationship. Contrary to prior studies, the present study provides atheoretical reconsideration of the concepts of performance evaluation styles. This
study categorizes performance evaluation styles by two dimensions: the priority ofbudgetary targets among the performance criteria at the stage of setting performance
criteria and the manner of using accounting information ex post. It also specifies twoconcepts to explain these dimensions, namely budget rigidity and discretionary
adjustments, under the assumption that environmental uncertainty determines the
optimal degree of each. The empirical results support this assumption; that is, thecombination of lower budget rigidity and higher discretionary adjustments is optimal
in a highly uncertain environment, while higher budget rigidity and higher
discretionary adjustments is optimal in a stable environment. Furthermore, the results
demonstrate that the active use of discretionary adjustments complements the positiveeffects of budget rigidity, regardless of environmental uncertainty.
However, the present study has limitations. First, the conceptual framework is
not completely original as it based on prior studies. Nevertheless, it helps to identify
the crucial dimensions to distinguish the performance evaluation styles.
The second limitation relates to the problems of variable measurement andempirical analysis. Endogeneity concerns cannot be eliminated because the study uses
a cross-sectional survey and cannot control several factors related to performance
evaluation styles, such as target difficulty and budget revisions. Because of theseoverlooked control variables, it is possible that omitted variable bias exists.
The third limitation is in the interpretation of the results of the empiricalanalysis. The results demonstrate that the combination of lower budget rigidity andhigher discretionary adjustments is suitable in a highly uncertain environment. Lowerbudget rigidity demonstrates the importance of emphasizing performance criteria
besides budgetary targets, but this study does not indicate what measure firms shouldemphasize in practice. Furthermore, the results cannot explain what information
supervisors use in their discretionary adjustments and how they evaluate managers’
performance. Future studies should investigate the concrete practices of performance
evaluation, which currently exist in a black box. In particular, qualitative researchmethods are important to uncovering the practice of performance evaluation accurately
because research on performance evaluation styles overemphasized quantitative
analyses.

21

Appendix
Survey instrument
Budget Rigidity
How do you implement budgetary control for the following? (1= not at all; 7=absolutely correct)
1. The business unit’s managers are constantly conscious of meeting budgetary targets.
2. Performance of business unit’s managers is assessed predominantly on the basis ofattaining budgetary targets.
3. Control over business unit is achieved by monitoring how well the budget’s targets
are on track.
4. Meeting the budget is an accurate reflection of whether managers are successful.
5. Not meeting the budget has a strong impact on managers’ performance evaluation.
6. Promotion prospects depend heavily on ability to meet the budget.
7. Not meeting the budget reflects poor performance.
Discretionary Adjustments
How does your company evaluate business unit managers? (1= not at all; 7=absolutely correct)
1. Budget target is evaluated subjectively based on change of conditions or explanation
by manager.
2. Quantitative non-financial targets are evaluated subjectively based on change ofconditions or explanation by manager.
Environmental Uncertainty
How accurately does your company predict the business environment 3 years ahead?
(1= very predictable; 7= very unpredictable)
1. Production, service, and information technologies
2. Diffusion of proprietary knowledge
3. Customer demand, tastes and preferences
4. Market activities of competitors
5. Emergence of new competitors
6. Suppliers’ actions
7. Industrial relations
8. Government regulation and policies

22

References
Aiken, L.S. and West, S.G. (1991), Multiple Regression: Testing and Interpreting
Interactions. Sage, London.
Aranda, C., Arellano, J., and Davila, A. (2019), “Subjective bonuses and target setting
in budget-based incentive contracts”, Management Accounting Research, Vol. 43,
pp. 45–60.
Argyris, C. (1952), The Impact of Budgets on People. The Controllership Foundation,
New York, NY.
Baker, G.P., Gibbons, R., and Murphy, K.J. (1994), “Subjective performance measuresin optimal incentive contracts”, Quarterly Journal of Economics, Vol. 109, No. 4,
pp. 1125–1156.
Banker, R.D. and Datar, S.M. (1989), “Sensitivity, precision, and linear aggregation of
signals for performance evaluation”, Journal of Accounting Research, Vol. 27, No.
1, pp. 21–39.
Becker, S.D., Mahlendorf, M.D., Schäffer, U., and Thaten, M. (2016), “Budgeting intimes of economic crisis”, Contemporary Accounting Research, Vol. 33 No. 4, pp.
1489–1517.
Bedford, D S. and Malmi, T. (2015), “Configurations of control: An exploratory
analysis”, Management Accounting Research, Vol. 27, pp. 2–26.
Bol, J.C. (2008), “Subjectivity in compensation contracting”, Journal of Accounting
Literature, Vol. 27, pp. 1–24.
Briers, M. and Hirst, M. (1990), “The role of budgetary information in performance
evaluation”, Accounting, Organizations and Society, Vol. 15, No. 4, pp. 373–398.
Brownell, P. (1982), “The role of accounting data in performance evaluation,budgetary participation and organizational effectiveness”, Journal of Accounting
Research, Vol. 20, No. 1, pp. 12–27.
Brownell, P. (1985), “Budgetary systems and the control of functionally differentiatedorganizational activities”, Journal of Accounting Research, Vol. 23, No. 2, pp.
502–512.
Brownell, P. and Hirst, M. (1986), “Reliance on accounting information, budgetary
participation, and task uncertainty: Tests of a three-way-interaction”, Journal of
Accounting Research, Vol. 24, No. 2, pp. 241–249.
Bruns, W.J. and Waterhouse, J.H. (1975), “Budgetary control and organizational
structure”, Journal of Accounting Research, Vol. 13, No. 2, pp. 177–203.
Derfuss, K. (2009), “The relationship of budgetary participation and reliance on23accounting performance measures with individual-level consequent variables: A
meta-analysis”, European Accounting Review, Vol. 18, No. 2, pp. 203–239.
Ekholm, B. and Wallin, J. (2011), “The impact of uncertainty and strategy on theperceived usefulness of fixed and flexible budgets”, Journal of Business Financeand Accounting, Vol. 38, No. 1/2, pp. 145–164.
Flamholtz, E. (1983), “Accounting, budgeting and control systems in theirorganizational context: Theoretical and empirical perspectives”, Accounting,
Organizations and Society, Vol. 8, No. 2–3, pp. 153–169.
Gibbs, M., Merchant, K., Van der Stede, W., and Vargus, M. (2004), “Determinants
and effects of subjectivity in incentives”, The Accounting Review, Vol. 79, No.2,
pp. 409–436.
Govindarajan, V. (1984), “Appropriateness of accounting data in performance
evaluation: An empirical examination of environmental uncertainty as anintervening variable”, Accounting, Organizations and Society, Vol. 9, No. 2, pp.
125–135.
Grabner, I. and Moers, F. (2013), “Management control as a system or a package?
Conceptual and empirical issues”, Accounting, Organizations and Society, Vol. 38,
No. 6–7, pp. 407–419.
Hartmann, F. (2000), “The appropriateness of RAPM: Toward the further developmentof theory”, Accounting, Organizations and Society, Vol. 25, pp. 451–482.
Hartmann, F., Naranjo-Gil, D., and Perego, P. (2010), “The effects of leadership styles
and use of performance measures on managerial work-related attitudes”,
European Accounting Review, Vol. 19, No. 2, pp. 275–310.
Hirst, M.K. (1981), “Accounting information and the evaluation of subordinate
performance: A situational approach”, The Accounting Review, Vol. 56, No. 4, pp.
771–784.
Hirst, M.K. (1983), “Reliance on accounting performance measures, task uncertainty,
and dysfunctional behaviour: Some extensions”, Journal of Accounting Research,
Vol. 21, No. 2, pp. 596–605.
Hofstede, G. (1967), The Game of Budget Control. Van Gorcum, Assen, Netherlands.
Holmstrom, B. (1979), “Moral hazard and observability”, Bell Journal of Economics,
Vol. 10, No. 1, pp. 74–91.
Hoppe, F. and Moers, F. (2011), “The choice of different types of subjectivity in CEO
annual bonus contracts”, The Accounting Review, Vol. 86, No. 6, pp. 2023–2045.
Hopwood, A.G. (1972), “An empirical study of the role of accounting data inperformance evaluation”, Journal of Accounting Research, Vol. 10, pp. 156–182.

24

Ittner, C.D., Larcker, D.F., and Randall, T. (2003), “Performance implications ofstrategic performance measurement in financial services firms”, Accounting,
Organizations and Society, Vol. 28, No. 7–8, pp. 715–741.
Libby, T. and Lindsay, R.M. (2010), “Beyond budgeting or budgeting reconsidered? A
survey of North-American budgeting practice”, Management Accounting
Research, Vol. 21, pp. 56–75.
Lillis, A.M. (2002), “Managing multiple dimensions of manufacturing performance:
An exploratory study”, Accounting, Organizations and Society, Vol. 27, No. 6, pp.
497–529.
Luft, J. and Shields, M.D. (2003), “Mapping management accounting: Graphics andguidelines for theory-consistent empirical research”, Accounting, Organizations
and Society, Vol. 28, pp. 169–249.
Marginson, D.W. and Ogden, S. (2005), “Coping with ambiguity through the budget:
The positive effects of budgetary targets on managers’ budgeting behaviours”,
Accounting, Organizations and Society, Vol. 30, pp. 435–456.
Merchant, K.A. (1981), “The design of the corporate budgeting system: Influences on
managerial behavior and performance”, The Accounting Review, Vol. 56, pp. 813–
829.
Merchant, K.A. (1990), “The effects of financial control on data manipulation andmanagement myopia”, Accounting, Organizations and Society, Vol. 15, No. 4, pp.
297–313.
Moers, F. (2006), “Performance measure properties and delegation”, The Accounting
Review, Vol. 81, No. 4, pp. 897–924.
Otley, D.T. (1978), “Budget use and managerial performance”, Journal of Accounting
Research, Vol. 16, pp. 122–149.
Otley, D.T. and Fakiolas, A. (2000), “Reliance on accounting performance measures:
Dead end or new beginning? ”, Accounting, Organizations and Society, Vol. 25, pp.
497–510.
Ross, A. (1995), “Job related tension, budget emphasis and uncertainty: A researchnote”, Management Accounting Research, Vol. 6, pp. 1–11.
Said, A.A., HassabElnaby, R.H., and Wier, B. (2003), “An empirical investigation of
the performance consequences of nonfinancial measures”, Journal of Management
Accounting Research, Vol. 15, pp. 193–223.
Sponem, S. and Lambert, C. (2016), “Exploring differences in budget characteristics,
roles and satisfaction: A configurational approach”, Management Accounting
Research, Vol. 30, pp. 47–61.

25

Sprinkle, G.B. (2003), “Perspectives on experimental research in managerial
accounting”, Accounting, Organizations and Society, Vol. 28, pp. 287–318.
Van der Stede, W.A. (2000), “The relationship between two consequences ofbudgetary controls: Budgetary slack creation and managerial short-termorientation”, Accounting, Organizations and Society, Vol. 25, pp. 609–622.

26