COEXISTENCE OFTEST EXECUTION EFFICIENCYANDTEST CASES ACCOUNTABILITY
Makoto Nakakuki

Noriyuki Mizuno

Kazuhiro Suzuki

Yoshinori Seino

TEF-DoTEF-DoTEF-DoTEF-DoSapporo, Hokkaido, Japan

Tomakomai, Hokkaido, Japan

Kawaguchi, Saitama, Japan

Sapporo, Hokkaido, Japan

maq.naq.go@gmail.com

mizu-nori@e-mail.jp

suzuki.kazuhiro@gmail.com

pek9dkl@gmail.com

22 April 2019, InSTA2019

1© 2019Test Engineer's Forum 北海道

Contents• Tendency of Written Test Cases• Analysis of Written Test Cases

22 April 2019, InSTA20192© 2019Test Engineer's Forum 北海道

TENDENCY OF
WRITTEN TEST CASES

22 April 2019, InSTA20193© 2019Test Engineer's Forum 北海道

Written test cases• Written test cases are used to achieve specific testingobjective.
• The test cases are also used for obtaining test steps toexecute them.
→These test cases are required to be executed efficiently.

22 April 2019, InSTA2019

4© 2019Test Engineer's Forum 北海道

Tendency of Test Cases Template• Written Test Cases are always described in specific testcases templates.
• These templates are used not only as lists that enable toconfirm software system, but also as lists of execution
test steps.

22 April 2019, InSTA2019

5© 2019Test Engineer's Forum 北海道

Tendency of Actual Written Test cases• The test case templates assist to generate written testcases.
• These templates cause the test cases focused on a
specific objective.

22 April 2019, InSTA2019

6© 2019Test Engineer's Forum 北海道

We call these test cases
as CPM test cases.

ExampleCPM Test Cases• Software specification documents are often simply copied,pasted and modified to generate this type of test cases.
• A CPM test cases often has only test cases description
without test step.
Test Case
Confirm reservation of songs
KARAOKE System can display reserved list of the songs
when user puts menu button on and select "Reserved list".
User can change the order of the songs and delete song
while multiple songs are reserved in KARAOKE System.
User can change the order of the songs and delete song by
Remote Controler or Main Unit of KARAOKE System.
Confirm Playing Option
User can add the playing option of reservation songs.
Following are playing options.
1. Key or the song, 2. Tempo of the Song
3. Setting of Guide Melody, 4. Setting of Guide Vocal
5. Sound Volume

22 April 2019, InSTA2019

7

Test Steps-© 2019Test Engineer's Forum 北海道

Example
Execution Efficiency Focused Test Cases• These test cases are especially focused on efficient testexecution.
• A execution efficiency focused test cases usually has
multiple test objectives.
Test Case
Test Steps
Confirm Playing Function of KARAOKE System
1. Reserve music
Display Playing State
2. Start playing music
1. Control [Tempo of the Song]2. Control [Sound Effect]
3. Control [Fast forward]
4. Control [Rewind]
Control Playing Option
5. Control [Jump to High Point of the Song]6. Control [Pause the Song]7. Control [Restart the Song]8. Control [Use Guite Vocals]
22 April 2019, InSTA2019

8© 2019Test Engineer's Forum 北海道

Example
Stepwise Designed Test Cases• Test suite composed of stepwise designed test cases isusually structured and organized.
Test Case
Reservation Contol
Reserve from the ID of the Songthe case the ID of
the song exists.

the case the ID of
the song doesn't exist.
Reserve from Result of Search

Test Steps- Start Karaoke System- Press "Menu" Button<Precondition: Menu State>
1. Press "Researve from the ID"
2. Input the ID of the Song3. Press "Reserve" Button
The same as above

the case of keyword searchusing valid Keyword- Start Karaoke System- Press "Menu" Button<Precondition: Menu State>
1. Press "Keyword Search" Button
2. Input the Keyword of the Song3. Press "Search" Button
4. Select one of the song.
5. Press "Reserve" Button

the case of keyword searchusing invalid Keyword

The same as above

22 April 2019, InSTA2019
the case of past ranking search<Precondition:
Menu State>
91. Press "Ranking Search" Button© 2019Test Engineer's Forum 北海道

Example
Stepwise Designed Test Cases

Test model like Fig isoften used while thistype of test cases
are designed.
22 April 2019, InSTA201910© 2019Test Engineer's Forum 北海道

ANALYSIS OFWRITTEN TEST CASES

22 April 2019, InSTA201911© 2019Test Engineer's Forum 北海道

Classification of Two Axis

Written Test Cases Positioning Map
22 April 2019, InSTA2019

12© 2019Test Engineer's Forum 北海道

Vertical Axis :Test Cases Accountability
Unaccountability
Accountability• The test suite composed of• The test suite composed of
unaccountable test cases isaccountable test cases is
not structured.
usually structured andstepwise designed.
• This characteristic oftencauses difficulty in describing • Abstract overview pictures
overview.
assist to review the test
cases.
• These unaccountable test
cases are usually difficult to• This type of test cases hasreview.
relatively less omissions.
• It is also hard to confirm• Test objectives of these test
traceability among test cases
cases are clarified.
and software specification
documents.
Test cases accountability means that test objectives is22 April 2019, InSTA2019

13 system owners and© 2019Test
Engineer's Forum 北海道
easy fortest operators.

Horizontal Axis :Test Execution Efficiency
Inefficient• Each test cases has only a
few test steps to confirm
one or a few test
objectives.
• Same test step is used
many times in the test
cases.

22 April 2019, InSTA2019

Efficient• Test cases has several test
steps for efficient test
execution.

14© 2019Test Engineer's Forum 北海道

Characteristics ofTest Cases inTesteacharea
Case AccountabilityThe written test
cases positioning
map has 4 areas.

Accountable
Area3:
Area4:
Accountable but
Accountable andinefficient Test Cases efficient Test Cases
Inefficient
Efficient
Test Execution
Area1:
Area2:
Efficiency
Unaccountable andUnaccountable but
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA2019

15© 2019Test Engineer's Forum 北海道

Characteristics of
the Area 1 Test Cases• Test Cases in this area areunaccountable andinefficient.
• The CPM test cases are
representative of test
cases in area1 of chapter.

Test Case Accountability
Accountable
Area3:
Area4:
Accountable but
Accountable andinefficient Test Cases efficient Test Cases
Inefficient
Efficient
Test Execution
Area1:
Area2:
Efficiency
Unaccountable andUnaccountable but
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA201916© 2019Test Engineer's Forum 北海道

Characteristics of
the Area 2 Test Cases• Test cases are efficient attest execution.
• A test cases in area 2 canbe integrated from several
test cases in area 1 or area
3.
• Multiple test objective are
mixed in a test cases.

Test Case Accountability
Accountable
Area3:
Area4:
Accountable but
Accountable andinefficient Test Cases efficient Test Cases
Inefficient
Efficient
Test Execution
Area1:
Area2:
Efficiency
Unaccountable andUnaccountable but
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA201917© 2019Test Engineer's Forum 北海道

Characteristics of
the Area 3 Test Cases• Test cases in this area
have characteristics ofaccountable test cases.
• The stepwise designed testcases are mapped in thisarea 3.

Test Case Accountability
Accountable
Area3:
Area4:
Accountable but
Accountable andinefficient Test Cases efficient Test Cases
Inefficient
Efficient
Test Execution
Area1:
Area2:
Efficiency
Unaccountable andUnaccountable but
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA201918© 2019Test Engineer's Forum 北海道

Characteristics of
the Area 4 Test Cases• Test cases in this area areaccountable and efficient.

Test Case Accountability
Accountable
Area3:
Area4:
Accountable but
Accountable andinefficient Test Cases efficient Test Cases
Inefficient
Efficient
Test Execution
Area1:
Area2:
Efficiency
Unaccountable andUnaccountable but
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA201919© 2019Test Engineer's Forum 北海道

Mapping of each Type of Test Cases
Fig shows eachtype of test cases
in mapped on thewritten test cases
positioning map.

Test Case Accountability
Stepwise
Accountable
Designed
Area3:
Area4:
Test Case but
Accountable
Accountable andinefficient Test Cases efficient Test Cases
Inefficient
Efficient
Test ExecutionCPM Test Case
Execution
Efficiency
Area1:
Area2:
Efficiency
Focused Test Case
Unaccountable andUnaccountablebut
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA201920© 2019Test Engineer's Forum 北海道

3 categories of improvement
We summarize 3
categories ofimprovement.

Test Case Accountability
Accountable
Area3:
Area4:
Accountable but
C Accountable andinefficient Test Cases
efficient Test Cases
Inefficient
EfficientBA
Test Execution
Area1:
Area2:
EfficiencyUnaccountable andUnaccountable but
Inefficient Test Cases efficient Test Cases
Unaccountable
Written Test Cases Positioning Map

22 April 2019, InSTA201921© 2019Test Engineer's Forum 北海道

To Achieve BETTER TEST CASES• We consider coexistence of test execution efficiency andtest suite accountability is ideal.
• Solutions for improvement from area 1 or area 2 test
cases to area 3 test cases have been already proposed toachieve better test cases.
• In the situation that test resource is sufficient, the area 3
test cases are also effective.

22 April 2019, InSTA201922© 2019Test Engineer's Forum 北海道

Special Thanks

specified nonprofit corporation
ASTER
Aster has been tolerant and supportive ...

ASTER:Association of Software Test EngineeRing
22 April 2019, InSTA201923© 2019Test Engineer's Forum 北海道

What is TEF-Do?
• Testing Engineer's Forum• Hokkaido software test study meeting• Also known as “TEF-Do”• It is formed by people who have relationship in Hokkaido.
• Activity• Study group• Interaction with other study group• Gathering
22 April 2019, InSTA201924© 2019Test Engineer's Forum 北海道