# Relative Priority Analysis of Korean IS Audit Standard Check Items Using the Constant-Sum Method

Boo-Hyoung Lee, Chi-Su Kim, Heon-Tag Kong

► To cite this version:

Boo-Hyoung Lee, Chi-Su Kim, Heon-Tag Kong. Relative Priority Analysis of Korean IS Audit Standard Check Items Using the Constant-Sum Method. 9th International Conference on Network and Parallel Computing (NPC), Sep 2012, Gwangju, South Korea. pp.233-240, 10.1007/978-3-642-35606-3_27. hal-01551367

HAL Id: hal-01551367

https://inria.hal.science/hal-01551367

Submitted on 30 Jun 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Relative Priority Analysis of Korean IS Audit Standard
Check Items Using the Constant-Sum Method

Boo-Hyoung Lee¹, Chi-Su Kim¹, Heon-Tag Kong¹

¹ Kongju National University, S.Korea

bhl1998@kongju.ac.kr

**Abstract.** Currently, our country have introduced IS audit system to manage and supervise stable development and efficient operation of the IS. IT audit is accomplished by IS auditor using standard check items made by our government. As IT paradigm drastically changes and user's IT service demands level is high, Flexible change or modification of standard check items is inevitable. It is reasonable to change or modify them according to the priority after establishing relative weight inter-standard check items. To do this, domestic standard check items are represented as hierarchical structure using Hierarchical Decision Model (HDM) and relative weights and priority among them are obtained by Constant-sum Method in this paper. This result will give a logical validity enough to modify or change domestic standard check items.

## 1 Introduction

As Information system (IS) is an integral part in all areas and the dependency on the system is getting larger, it is very important to predict and prevent risks that may take place in developing and operating an information system [1]. The Korean government have introduced the IS Audit System to manage and supervise the stable development and effective operation of IS since July 2006. Also, it is stipulated that every IS development companies which develop programs over 0.5billion ordered by government branches must be IS audited [2].

The IS Audit is defined as activities conducted by an authorized person for the purpose of providing an independent assessment of software products and processes in order to assess compliance with requirements. In general, the auditing is executed based on "The standard check items" by auditing-companies registered in the Minister of Public Administration and Security. After IS audit, the auditing companies should submit the audit results which follows specific rules. The auditing-company must be independent with the audit ordering agency and auditee. The audit ordering agency indicates government branches that order the IS development and the auditee means IS development company [3].

Since the initiation of the Information System Audit at 1987 by National Information society Agency, the foundation of private audit corporations was permitted in 1998 and IS audit was activated. The basis for the Information System Audit is added in Electronic Government Act(Art. 57), and Electronic Government Act Enforcement Decree(Art. 71). According to it, the Ministry of Public Administration and Security released "Information System Audit standards" which stipulates the audit procedures and methods for effective implementation of audit tasks including audit agreement, audit planning, onsite audit, audit report preparation, and inspection of audit result applications. According to Art. 23 in it, National Information society Agency published "Practical Guide for IS Audit" which defines the detailed audit procedures and methods, standard check items, format of IS audit plan and report etc[4].
Although IS Audit is executed by such mandatory audit policy and need of it is increased, there are still many negative opinions regarding the effect and quality of the audit. Typical complains are subjective audit results that cannot be agreed or audit results not helping management decision making. There are many studies to solve such problems and enhance audit quality [5-7].

Tae-Won Kyung et al released "a Study on the Priority Analysis of Information System Audit Service Evaluation Items by Means of Fuzzy AHP Technique," where he defines the Information System Audit as a service activity, highlights the factors to differentiate and improve the quality of audit services, and endeavors to quantify the priority of such factors[8].

Don-ik Shin suggests the two step auditing method - the process audit and then product audit - in his "a Scientific Method for Information System Audit," and claims the necessity to improve the validity and reliability of audit by means of this method[9].

Gi-chan Nam et al, in his "a Comparative Analysis of Information System Audit and International IT Guidance," analyzes the correlation between the current Audit Standards and international IT guidances, and presents the enhanced audit standard in compliance with the international IT guidance[10].

The above studies are focused on only methods such as introduction of evaluation methods for audit results, improvement of auditor ability and change of the auditing method to improve audit quality. As long as the current audit standards check items continue to be applied in auditing in spite of IT paradigm drastically changes, the system development environment is consistently changing and user demands are becoming more stringent, it is difficult to improve audit quality.

In order to modify or change audit standards check items, it is necessary to prioritize the current audit check items by experts, and then modify or change them based on the priority.

To do this, standard check items are presented in a hierarchical structure by means of HDM method, one of the decision-making methods and then the relative weights among standard check items are measured by means of the Constant-Sum Method. Finally, the priority of each item are drawn out. These results make valid basis for modification and change of audit standards that can improve audit quality.

## 2 Korean IS Audit Check Standard

As stated in the introduction, the fundamental directions for the Information System Audit is based on "Practical Guide for IS Audit" published by the National Information Society Agency. Especially, the development systems are assessed according to 'Standard check items' in "Practical Guide for IS Audit". The standard check items have the structure of project type based check system. Accordingly, the standard check items have check items which should be assessed by project type. The project type is classified into 6 areas- IT Architecture, IT Strategy, IS System Development, DB Construction, System Operation and Maintenance. In view point of auditing, the project type is considered as the audit field. Each audit field is divided into 3 audit times (phase), 6 audit sub-areas and detailed check items [11]. Table 1 shows the project type based standard check items.
<table><caption>**Table 1** The project type based standard check items</caption><thead><tr><th>Audit Field</th><th>Audit Time</th><th>Sub-area</th><th>Detailed Audit items</th></tr></thead><tbody><tr><td rowspan="6">IT Architecture</td><td rowspan="3">Foundation Set-up& Current Architecture Construction</td><td>Foundation Set-up</td><td>4 items</td></tr><tr><td>Current Architecture Construction</td><td>9 items</td></tr><tr><td>Quality Control</td><td>4 items</td></tr><tr><td rowspan="3">Goal Architecture Construction & Implementation Plan</td><td>Goal Architecture Construction</td><td>6 items</td></tr><tr><td>Implementation Plan</td><td>4 items</td></tr><tr><td>Management</td><td>4 items</td></tr><tr><td rowspan="6">IT Strategy</td><td rowspan="3">Current step analysis & Strategy development</td><td>Quality Control</td><td>4 items</td></tr><tr><td>Task Analysis</td><td>7 items</td></tr><tr><td>Technology Analysis</td><td>5 items</td></tr><tr><td rowspan="2">Improvement Model & Implementation plan</td><td>Quality Control</td><td>3 items</td></tr><tr><td>IT plan</td><td>9 items</td></tr><tr><td></td><td>Quality Control</td><td>3 items</td></tr><tr><td rowspan="12">IS system Development</td><td rowspan="3">Demand Analysis</td><td>System Architecture</td><td>4 items</td></tr><tr><td>Applied System</td><td>5 items</td></tr><tr><td>Database</td><td>3 items</td></tr><tr><td rowspan="3">Analysis/Design</td><td>System Architecture</td><td>5 items</td></tr><tr><td>Applied System</td><td>9 items</td></tr><tr><td>Database</td><td>6 items</td></tr><tr><td rowspan="3">Implementation</td><td>System Architecture</td><td>3 items</td></tr><tr><td>Applied System</td><td>6 items</td></tr><tr><td>Database</td><td>4 items</td></tr><tr><td>Testing</td><td>Test Activities</td><td>7 items</td></tr><tr><td>Deployment</td><td>Preparation for Operation</td><td>4 items</td></tr><tr><td rowspan="3">DB Construction</td><td rowspan="2">Preparation</td><td>Data Collection Model Construction</td><td>7 items</td></tr><tr><td>Data Construction</td><td>4 items</td></tr><tr><td>Implementation</td><td>Quality Control</td><td>3 items</td></tr><tr><td rowspan="2">System Operation</td><td rowspan="2">Operation</td><td>User-Service</td><td>8 items</td></tr><tr><td>User-Service support</td><td>6 items</td></tr><tr><td>Maintenance</td><td>Maintenance</td><td>Maintenance</td><td>6 items</td></tr></tbody></table>
## 3 Establishment of the Relative Weights inter-standard check items

This study aims to decide the relative weights among the domestic audit standard check items, and determine the priority. To this end, the Hierarchical Decision Model which is decision-making technique is adapted. The hierarchical decision making is the technique that divides into hierarchical steps [12]. The decision-making using the Hierarchical Decision Model is as follows:

First, All elements involved in the decision-making enlists and stratifies by the Hierarchical Decision Model

Second, 1:1 pair-wise comparison is conducted among elements on the same level in each hierarchical step and score(100 point measure) is given to each element.

Third, scores are analyzed to decide the relative weights of each element and prioritize them.

The representative Hierarchical Decision methods are The Analytic Hierarchy Process (AHP) and Hierarchical Decision Model (HDM) suggested by Thomas L. Saaty and Dundar F. Kocaoglu, [13][14]. These are the same in that they presents all elements hierarchically, but in the way to obtain the relative weights or priority of items in each hierarchical step, Saaty adopts the 'Eigenvalue method' while Dundar adopts the 'Constant Sum method.' Kocaoglu's HDM and the Constant Sum Method are used in this paper.

### 3.1 Hierarchical Decision Model (HDM)

HDM Hierarchical Decision Model represents elements to be decided as in Fig. 1 in the three levels - Impact, Targe, and Operational. More specifically, it includes MOGSA(Mission/ Objectives/ Goal/ Strategy/ Action) stages. The Impact Level, as the highest stage, is the ultimate goal of decision-making and consists of Mission/ Objectives. The Target Level is the goal level and consists of methodological elements to carry out missions of the highest level. The Operational Level is the level including Strategy/ Action and has more specific activities or alternatives that may affect the mission or the ultimate goal. Elements in the lower level are comparatively specific, and each element on the same level should be comparable with each other.
<figure><img src="image_2.png" /><figcaption>Fig.3 A Typical Hierarchical Decision Model</figcaption></figure>

## 3.2 Constant-Sum Method

To give weight value to elements in each level, 1 to 1 pairwise comparison is conducted inter-elements in each level, and the extent of contribution to the upper-level elements is expressed as the 100 point measure by the Constant-Sum Method. For example, if the relative weight of one element is 4 times higher than that of the other element, then they are given 80 points and 20 points respectively. If the relative weight of one element is a bit higher than that of the other, they are given 52 points and 48 points respectively. If two elements have the same relative weight, then they are given 50 points respectively. Once every element is given the score after comparison, the final matrixes A, B, and C are calculated based on the assigned scores, and the priority is decided [14].

# 4 Data Collection and Analysis Results

To measure the relative weights and priority of standard audit items, the standard check items are presented as a hierarchical structure by the Hierarchical Decision Model and the pair wise comparison is conducted among elements in the same level to grade each element. The score by the pair wise comparison was obtained through the survey of IS experts. In this study, the standard check items are represented in the 3-stage Hierarchical Decision Structure of HDM for each project type, the survey is conducted to grade items based on the pair wise comparison. 21 IS experts participated in the survey. The scores given by the experts are analyzed to measure the relative weights inter-audit items.
## 4.1 HDM Structure of Standard Check items

Korean standard audit items can be represented in the three-stage hierarchical decision model. Fig. 2 shows an example of the hierarchical decision model for standard check items on the IT Architecture in Table 1. In Fig. 2, the IT Architecture consists of the two audit time, 7 audit sub-areas, and 35 detailed check items.

<figure><img src="image_3.png" /><figcaption>Fig. 2 An example of the hierarchical decision model for IT Architecture</figcaption></figure>

## 4.2 Results

The relative weights inter-standard check items were measured by means of a pairwise comparison survey conducted by IS experts. Pair-wise comparison was done for elements on audit time and sub-area in each audit field. Among the 21 participants in the survey, 11 are working at IS auditor, and the other 10 were IT experts. Table 2 presents the relative weights of inter-elements on audit time and sub-area in each audit field

<table><thead><tr><th>Audit Field</th><th>Audit Time</th><th>Weight</th><th>Sub-area</th><th>Weight</th><th>Final weight</th><th>Pri ority</th></tr></thead><tbody><tr><td rowspan="10">IT Architecture</td><td rowspan="3">Foundation Set-up &amp; Current Architecture Construction</td><td rowspan="3">0.55</td><td>Foundation Set-up</td><td>0.38</td><td>0.22</td><td>1</td></tr><tr><td>Current Architecture Construction</td><td>0.34</td><td>0.18</td><td>2</td></tr><tr><td>Quality Control</td><td>0.27</td><td>0.15</td><td>3</td></tr><tr><td rowspan="4">Goal Architecture Construction &amp; Implementation Plan</td><td rowspan="4">0.45</td><td>Goal Architecture Construction</td><td>0.29</td><td>0.13</td><td>1</td></tr><tr><td>Implementation Plan</td><td>0.26</td><td>0.12</td><td>2</td></tr><tr><td>Management</td><td>0.23</td><td>0.10</td><td>3</td></tr><tr><td>Quality Control</td><td>0.21</td><td>0.09</td><td>4</td></tr><tr><td>IT Strategy</td><td>Current step</td><td>0.59</td><td>Task Analysis</td><td>0.44</td><td>0.26</td><td>1</td></tr></tbody></table>
<table><tbody><tr><td rowspan="10"></td><td rowspan="3">analysis &amp; Strategy development</td><td></td><td>Technology Analysis</td><td>0.31</td><td>0.18</td><td>2</td></tr><tr><td rowspan="2">Improvement Model &amp; Implementation plan</td><td></td><td>Quality Control</td><td>0.26</td><td>0.15</td><td>3</td></tr><tr><td>0.41</td><td>IT plan</td><td>0.63</td><td>0.26</td><td>2</td></tr><tr><td></td><td></td><td>Quality Control</td><td>0.37</td><td>0.37</td><td>1</td></tr><tr><td rowspan="12">IS system Development</td><td rowspan="3">Demand Analysis</td><td rowspan="3">0.33</td><td>System Architecture</td><td>0.38</td><td>0.13</td><td>1</td></tr><tr><td>Applied System</td><td>0.29</td><td>0.10</td><td>3</td></tr><tr><td>Database</td><td>0.33</td><td>0.11</td><td>2</td></tr><tr><td rowspan="3">Analysis/Design</td><td rowspan="3">0.24</td><td>System Architecture</td><td>0.37</td><td>0.09</td><td>1</td></tr><tr><td>Applied System</td><td>0.29</td><td>0.07</td><td>3</td></tr><tr><td>Database</td><td>0.33</td><td>0.08</td><td>2</td></tr><tr><td rowspan="3">Implementation</td><td rowspan="3">0.15</td><td>System Architecture</td><td>0.20</td><td>0.03</td><td>3</td></tr><tr><td>Applied System</td><td>0.32</td><td>0.05</td><td>2</td></tr><tr><td>Database</td><td>0.48</td><td>0.07</td><td>1</td></tr><tr><td>Testing</td><td>0.15</td><td>Test Activities</td><td>*</td><td>0.15</td><td>1</td></tr><tr><td>Deployment</td><td>0.13</td><td>Preparation for Operation</td><td>*</td><td>0.13</td><td>2</td></tr><tr><td rowspan="4">DB Construction</td><td rowspan="2">Preparation</td><td rowspan="2">0.65</td><td>Data Collection Model Construction</td><td>*</td><td>0.65</td><td>1</td></tr><tr><td>Data Construction</td><td>0.56</td><td>0.20</td><td>2</td></tr><tr><td rowspan="2">Implementation</td><td rowspan="2">0.35</td><td>Quality Control</td><td>0.44</td><td>0.15</td><td>3</td></tr><tr><td>User-Service</td><td>0.54</td><td>0.54</td><td>1</td></tr><tr><td>System Operation</td><td>Operation</td><td>*</td><td>User-Service support</td><td>0.46</td><td>0.46</td><td>2</td></tr><tr><td>Maintenance</td><td>Maintenance</td><td>*</td><td>Maintenance</td><td>*</td><td>*</td><td></td></tr></tbody></table>

In table 2, relative weights of inter-element in audit time and sub-area is shown. The final weights are calculated by the relative weight of the certain item multiplied by the upper level item's relative weight. For the project type such as System operation and Maintenance have only one element, it was unable to calculate the relative weight since a pair-wise comparison could not be conducted for one element. So it is recorded as *. After final weights are obtained, the priority is calculated based on it. This result gives a logical validity enough to modify or change standard check items.

## 5 Conclusion

The IS Audit is defined as activities conducted by an authorized person for the purpose of providing an independent assessment of software products and processes
in order to assess compliance with requirements. In general, the auditing is executed based on "The standard check items" by auditing-companies registered in the Minister of Public Administration and Security. Although such activity contributes to protecting computer system assets, maintaining the integrity of data and securing the efficiency and effectiveness of the system to a large extent, there have been many negative views on the audit effect and quality. Although there have been a lot of studies on audit effect and quality, most of them focus on improvement of auditor ability human resources, problem-solving for poor audit companies, and quality evaluation for audit results. However, it is essential to flexibly change the audit check items in order to improve the audit quality as information technologies become converged and complex, various technologies emerge, and software becomes more intelligent, larger, and more complicated. To this end, this study calculated relative weight and priority of inter-standard check items using HDM which is a Hierarchical Decision-making Model and the Constant-Sum Method which is a method to give relative weight score. These results will give a logical validity enough to modify or change standard check items.

## References

1. National Computerization Agency, "A Study on the Audit Programs for Information System the Report of 2004 R&D Results", 2004

2. Electronic Government Act Art. 57(Information System Audit for Administrative Agencies, etc), Enforcement Decree Art 71 of Electronic Government Law(Information System Audit),

3. Announcement from the Ministry of Public Administration and Security (No, 2010-85), Revision of the Information System Audit Standard

4. National Information Society, "Informationalization Project Management Manual(ver. 1.0-2011.07)」

5. The Direction of the Information System Audit System, Information System Audit Symposium, 2009

6. The Direction of ICT Audit in IT Convergence, Information System Audit Symposium, 2010

7. Measures for the Information System Audit in the Future Computing Environment, Information System Audit Symposium, 2011

8. Tae-won Gyeong and Sang-guk Kim, "A Study on the Priority Analysis of Information Systems Audit Evaluation Factors using Fuzzy-AHP Method", Information Systems Review, Vol. 10, No. 3 pp.155-183, 2008.

9. Dong-ik Shin, "A Scientific Method of Information System Audit", Informationalization Policy Volume No.15, 2nd edition, pp. 3-24, Summer Edition, 2008

10. Gichan Nam et al, "A Study on Comparative Analysis for Information Systems Audit and International IT Guidance", the final report of issues entrusted to the National Information Society, 2008, 12.

11. Information System Audit Comments Ver. 3.0, National Information Society Agency, 2008.

12. Geun-tae Jo, Yong-gon Jo, Hyeons-su Kang, "Analytic Hierarchy Process", Donghyun Publishing Co., 2003.

13. Saaty, T. L., "The Analytic Hierarchy Process", McGraw-Hill, New York, 1980.

14. Dundar F. Kocaoglu, "A participative Approach to Program Evaluation", IEEE Transaction on Engineering Management, Vol. Em-30, No, 3, Aug. 1983.