1Systematic Generation of Software Test Items
Based on System Behavior from User's Viewpoint

Masayuki Hirayama, Tetsuya Yamamoto

R&D Center System Engineering Lab., TOSHIBA Corporation
1 Komukai Toshiba-cho, Saiwai-ku, Kawasaki, 212-8582, Japan
masayuki.hirayama@toshiba.co.jp
Takuya Kishimoto

R&D Group, Home Appliance Company, TOSHIBA Corporation
Osamu Mizuno, Tohru Kikuno

Dept. of Informatics and Mathematical Science, Osaka University
Abstract

This paper proposes a new design method for software test, which willproduce test cases based on Use-Case Analysis, and then generate test items systematically using Software Fault Tree Analysis. The attractive features of theproposed method are summarized as follows: (1) use cases are described fromthe end users' viewpoint, (2) Illegal behaviors on the use-case description aretransformed into test items to be checked, and (3) test items are generated systematically without their duplication. The experimental application to defrosting
subsystem of refrigerator con rms the e ectiveness of the proposed method.
1.

Introduction

Recently, many computer systems or software systems are introduced into
our social or daily life. For example, most of electrical appliances have embedded
software in them. If software faults are remained in an electrical appliance and theelectrical appliance suddenly don't work correctly, then the scope of its e ect ordamage may become enormously large. Therefore even for electrical appliances,
ensuring the reliability of software becomes an essential requirement[6].
When an embedded software in electrical appliances is relatively small insize, the formal method based on nite state machine is used e ectively[2]. The
test items for legal behaviors are generated using transition sequences on the nitestate machine. On the contrary, test items for illegal behaviors are successfully
derived and checked carefully by experienced developers.
PSAM5c 2000 by Universal Academy Press, Inc.

2Since customers' demands for electrical appliances have increased rapidly,
the size of software has already become large. Then the formal approach doesn't
work e ectively in the development. The formal method can be applied only tothe core part of the system.
In this paper, we propose a new method for generating test items to checkillegal behaviors in the development of electrical appliances. The proposed method
has the following attractive features.
(1) Description from users' point of view: In order to describe the target system's behavior from user's point of view(rather than developer's point ofview), the use case description proposed by UML[1] is applied.
(2) Systematic detection of illegal behaviors: In order to detect systematicallythe main factors of illegal behaviors of the target system, the deviation
analysis[3, 4] is performed on the use case description.
(3) Stepwise generation of test items: The software fault tree[2, 6] for the target
system is constructed based on the deviation analysis, and then test items
for checking illegal behaviors are generated stepwisely from the fault tree.
Similar approach has already been proposed for general software by Smidts
et al.[5]. They implemented an architecture based software reliability model andextensively used fault trees in the analysis. But it seems to be dicult to apply
their method to the development of electrical appliances.
2.

New Design Method

The proposed method consists of the following ve steps:
Step-1 (System behavior understanding): We describe the software block-diagramand hardware block-diagram. By doing this, we can understand an outlineof the functional behavior of the target software system.

Step-2 (Use case analysis): We describe typical behavior of the target software

using activity chart and clarify important reliability factors by applying
use case analysis. Use case description and analysis are borrowed from theobject-oriented developing methodology (UML).

Step-3 (Deviation analysis): According to guide words, we extract unusual sit-uations in the use case description and nd such operations that deviate
from the basic behavior and cause abnormalities.

3

Step-4 (Software fault tree construction): We analyze the situations that bringundesirable illegal behaviors by referring to analysis results and use case description. We then successively consider the internal processing of software,and nally construct a kind of fault tree.

Step-5 (Test item generation): By extracting the factor on the leaf of the soft-ware fault tree, we generate test items that check this factor.

Furthermore, we construct a keyword dictionary and place such restriction
on the words in Step 2, Step 3 and Step 4 that they must be taken from thekeyword dictionary. By this restriction, we can avoid miscommunication among
Step 2, Step 3 and Step 4.
3.

Application to Refrigerator

In order to con rm the e ectiveness, we have analyzed real developmentof refrigerator. In the analysis, we consider software for a defrosting subsystem(See Figure 1). The refrigerator has two cooling rooms: a freezing room and arefrigerating room. The defrosting function is constructed by two basic activating
operations for the defrosting heater and defrosting fan, and it is controlled basedon the states of two rooms, elapsed time of defrosting, and rooms' conditions.
At Step-1, the engineer reads carefully the given functional speci cation tounderstand the functions of the defroster unit, and then describes software block
diagrams for the defroster.

88 Defrosting timer>>>< Defrosting fan>Freezingroom>
heater>><: DefrostingRoom
thermometer
8 Defrosting timer
Refrigerator system >>>< Defrosting fan>Refrigeratingroom>
heater>>:: Defrosting
Room thermometer
Fig. 1.

Organization of refrigerator system

At Step-2, the use case description for the defroster function is describedbased on the software block diagrams and the functional speci cation. First, thetop-level layer of the use case description(shown in Figure 2(a)) is described bytracing the block diagrams roughly. The transition between activities is represented by two kinds of arrows: 0! and |{. An arrow 0! shows a usual or4

<Checking | Failure>

Check
Check condition
conditionforfor defrosting
defrosting<Defrosting| Failure>
DefrostDefrost thetherefrigeratingrefrigerating roomroom

DefrostDefrost thethefreezing
freezing room
room

<Defrosting | Failure>(a) Top Layer
Check
Check history
history ofof
R/F
R/F defrosting
defrosting<Checking | Failure>
In initial behavior<Defrosting history | Exist><Defrosting history| Not exist>
Check<Checking
Check elapsed
elapsed time
time ofof
freezing| Failure>freezing room
room

Count
Count elapsed
elapsed time
time ofof
freezing
freezing room
room

<F-Timer | Expire (5.0h)><Checking| Failure> Check
Check elapsed
elapsed time
time ofof
refrigerating
refrigerating roomroom<R-Timer| Expire (5.0h)>

Defrost the refrigerating room

Defrost the freezing room(b) 2nd Layer
Fig. 2.

Use case analysis

legal behavior, but the other arrow |{ shows an unusual or illegal behavior. The
engineers then described a detailed description. For example, the activity \Checkcondition for defrosting" in Figure 2(a) is extended into the detailed description
shown in Figure 2(b) using the functional speci cation.
At Step-3, the illegal behaviors of the defroster are extracted by tracing theactivity chart shown in Figure 2. The guide words, prepared for the experiment,
support to extract the detail of an illegal behavior. For example, consider theactivity \Check history of R/F defrosting." The guide words for the \history"are \not exist" and \be incorrect," and the guide word for \check" is \be notactivated." These guide words are extensively used in Step-4.
At Step-4, by tracing the activity chart in Figure 2, the software fault
tree shown in Figure 3 is constructed as follows: First, the root node is determined. Since the most crucial failure for the defroster is that the frost cannot
be removed, the failure \Refrigerator cannot defrost" is adopted as the root. According to the top layer description, three nodes \Failure in checking condition
for defrosting," \Failure in defrosting freezing room" and \Failure in defrosting
refrigerating room" are chosen as the next level nodes of the software fault tree.
Then, for the \Failure in checking condition for defrosting," the nodes \Failurein checking history of R/F defrosting," \Failure in checking elapsed time of Froom" and so on are derived as the next level nodes. Then, using the result of
the deviation analysis, each node is expanded to some nodes successively.
At Step-5, test items are derived from the leaves and interior nodes of thesoftware fault tree. From the leaves with light gray in Figure 3, the test items

5

Refrigerator
cannot defrost
Failure in checking
condition for defrosting

Failure in defrosting
the freezing room

Failure inchecking historyof R/F defrosting

Failure inchecking elapsed
time of F-room

The history
record does notexist.

Checking thehistory record
is not activated.

F-room timer isnot activated.

Failure in defrosting
refrigerating room

Failure inchecking elapsed
time of R-room

The history
record isincorrect.

Timer data
is lost.

Fig. 3.

Timer data
is incorrect.

Timer is not
set as 5hours.

Timer speed
was too fast.

Timer speed
was too slow.

Software fault tree

A1.1 to A3.4 shown in Figure 4 are generated.
4.

Experimental Evaluation

The following were concluded from an experimental application of theproposed method to the refrigerator control software.
(1) With relatively a few additional e ort, we can generate test items suitable
to check behaviors (especially, illegal behaviors) of the target system.
(2) The proposed method generates test items in more detailed descriptions,which re ects and thus corresponds to the implementation of the targetsoftware.
(3) The proposed method generates systematically all necessary test items without omission(if a given speci cation is well written and complete).
5.

Discussion

In software testing, checking of such illegal behaviors that were overlookedat test design needs a lot of extra costs. Generally speaking, essential functions of
the products would be occasionally overlooked when user's viewpoint comes rst.
But, since the proposed method is currently developed only for embedded software
in electrical appliances and the fault tree analysis is applied at the nal stage ofthe proposed method, such defects may be avoided and test items generated by6Condition
A1 Switch on
defrosting
A2 Checking
history of R/F
defrosting

Test Items
A1.1 Is history checking activated?
A2.1 Does defrosting history exist?
A2.2 Is defrosting history correct?
A3.1 Is F-room timer activated?
A3.2 Does F-room timer count?
A3.3 Does F-room timer count correctly?

A3Initial defrosting
after switch on

A3.4 Is F-room timer set for5 hours?
A3.5 Is neither F- nor R-defroster activated, so far as F- andR-room timer < 5 hours?
A3.6 Is F or R-defroster activated, so far as timer count islarger than 5 hours?
......
......

......

A8.1 Is the remaining time correct?
A8
Defrosting timer >6 hours

A8.2 Is the timer set by half of the remaining time?
......

A9 Check forresidual ice......

Fig. 4.

Generated test itemsthe proposed method cover all essential functions. Thus, as a result, we can expect
test costs might be shortened compared with the one by conventional method.
References[1] H. -E. Eriksson and M. Penker, UML toolkit, John-Wiley & sons, 1997.
[2] T. Fukaya, M. Hirayama and Y. Mihara, \Software speci cation veri cation
using FTA," Proc. of FTCS-24, pp.131{133, 1994.
[3] N. G. Leveson, Safeware: System safety and computers, Addison-Wesley,
1995.
[4] J. D. Reese, et al., \Software deviation analysis," Proc. of 19th International Conference on Software Engineering(ICSE'97), pp.250{260, 1997.
[5] C. Smidts and D. Sova, \An Architectural Model for Software Reliability Quanti cation: Sources of Data," Reliability Engineering and System
Safety, vol.64, 279{290, 1999.
[6] K. Tamura, J. Okayasu and M. Hirayama, \A software testing method
based on hazard analysis and planning," Proc. of ISSRE'98, pp.103{110,1998.