Classification Tree Method with Parameter Shielding
Takashi Kitamura1 , Akihisa Yamada2 , Goro Hatayama3 ,Shinya Sakuragi3 , Eun-Hye Choi1 , Cyrille Artho4
1

Nat. Inst. of Advanced Industrial Science and Technology (AIST), Japan{t.kitamura, e.choi}@aist.go.jp
2
Innsbruck University, Austria akihisa.yamada@uibk.ac.at
3
Omron Social Solutions Co., Ltd., Japan{goro hatayama,shinya sakuragi}@oss.omron.co.jp
4
KTH Royal Institute of Technology, Sweden artho@kth.se

Abstract. The Classification Tree Method (CTM) is a structured and diagrammatic modeling technique for combinatorial testing. CTM can express the notionof “parameter shielding”, the phenomenon that some system parameters become
invalidated depending on another system parameter. The current form of CTM,
however, is limited in its expressiveness: it can only express parameter shieldingthat depends on a single parameter. In this paper, we extend CTM with parametershielding that depends on multiple parameters, proposing CTMshield . We evaluate
the proposed extension on several industrial systems. The evaluation finds thatparameter shielding often depends on multiple parameters in real systems, andthe effectiveness of the extension.

1

Introduction

Testing is an important and often a necessary system development process for assuringsystem quality in current industrial practice. Combinatorial testing is a system testing
technique, that effectively tests the interactions of parameters in a system under test.
Combinatorial testing derives, typically from specification, a test model, which consistsof a list of parameter-values and constraints over them. Based on such test models, test
suites are designed, that consider various coverage criteria, such as t-way testing [1] [2].
Fig. 1 shows an example test model, which specifies an IC card system with six
parameters, each having two to three values: the Age of the card owner, the Balancethat is already charged in the card, whether Credit Card (C.C.) information is available
or not, the Charge Method (C.M.) and Charge Amount (C.A.) the owner specifies tothe system, and the Monthly Total (M.T.) amount of usage. The model also indicatesconstraints in logic formula, specifying valid (and invalid) value combinations. The
two constraints in the example model specify the following specifications:– “An IC card owned by a child cannot have Credit Card information.”– “The Charge Method can be by credit card only if a Credit Card information isavailable.”
Table 1 shows, as a test suite example, a 2-way test suite of the test model.

PARAMETERSVALUES
Agechild, adult, senior
Balance>190e, ≤190e
Credit Card (C.C.)
with, without (w/o)
Charge Method (C.M.) cash, credit card (c.c.)Charge Amount (C.A.) 10e, 50e
Monthly Total (M.T.) >390e, ≤390e

CONSTRAINTS¬(C.C. = with ∧ Age = child)(C.M. = c.c. ⇒ C.C. = with)

Fig. 1: A test model example for an IC card system; it consists of a parameter-values
list (left) and a set of constraints (right).

Table 1: A 2-way test suite for the test model of Fig. 1, which covers all valid value
pairs but avoids invalid ones, e.g. hC.C.=with, Age=childi, specified by the constraints.
No. Age Balance C.C. M.T. C.M. C.A.
1 child >190e w/o ≤390e cash 10e
2 child ≤190e w/o >390e cash 50e
3 adult >190e w/o ≤390e cash 50e
4 adult ≤190e with >390e c.c. 10e
5 senior >190e with ≤390e c.c. 50e
6 senior ≤190e w/o ≤390e cash 50e
7 senior >190e with >390e cash 10e

A key challenge in applying combinatorial testing in real-world development ismodeling, a. k. a. Input Parameter Modeling [2] or Input Domain Modeling [3]. Models in real-world systems often involve complex constraints on parameter-values. This
makes modeling a time-consuming and error-prone task that requires experience andcreativity of test experts.
Classification Tree Method (CTM) [4,5,6] is a structured and diagrammatic approach for the modeling problem. The main characteristic of CTM is that, using atree-structured modeling language called Classification Trees (CTs), is to be able todescribe the notion of “Parameter shielding” concisely, which is a phenomenon thatsome parameters become invalided (i. e., shielded) if some specific values are (or arenot) assigned to another parameter.
Suppose, for instance, the following specification SPEC1 is added to the system:
SPEC1: “Charging is allowed only if the Balance is below 190e.”Fig. 2 shows a CTM model that expresses this specification using a tree structure. The
tree structure expresses not only that (1) the relation between parameters and values, butalso (2) compositions of parameters and (3) parameter shielding. The rounded rectangle
node Charge combines Charge Method and Charge Amount of the previous example,
and appears under value ≤190e of Balance. This expresses that the two parameters
become valid only when the Balance is below 190e, and become invalid (shielded)
otherwise. Table 2 shows a 2-way test suite for the CTM model. Note that some parameters are assigned the vain value “—” when they are invalid. Note also that the test suitecharge IC card

C.C.

Balancewithout
with

Age

child

>190€

senioradult

Monthly Total

CONSTRAINTS:・ ￢ (C.C.=with ⋀ Age=child)>390€

・ C.M.=c.c. ⇒ C.C.=with

≦190€

≦390€

ChargeMethodcash c.c.

Amount10€50€

Fig. 2: A CTM test model for the IC card system that expresses SPEC1.

Table 2: A 2-way test suite for the model of Fig. 2 under SPEC1, where parameters
C.M. and C.A. are shielded (as assigned the vain value ‘—’) when Balance is >190e.
No. Age Balance C.C. M.T. C.M. C.A.
1 child ≤190e w/o ≤390e cash 10e
2 child ≤190e w/o >390e cash 50e
3 child >190e w/o >390e — —4 adult ≤190e with >390e c.c. 10e
5 adult ≤190e w/o ≤390e cash 50e
6 adult >190e with ≤390e — —7 senior ≤190e with >390e cash 10e
8 senior ≤190e with ≤390e c.c. 50e
9 senior >190e w/o ≤390e — —of Table 1 is not a valid 2-way test suite for the current model anymore, since, e.g., test
case No. 1 in Table 1 is not executable under SPEC1.
Parameter shielding expressed in a tree structure is a unique and useful feature of
CTM; however, its limitation is that it can only describe parameter shielding that depends on a single parameter-value. The reason is obvious: the dependencies of parameter shielding are expressed within the tree structure, and hence a parameter can only
have one parent. In our case studies applying combinatorial testing and CTM to industrial systems, however, we often encountered a demand to express parameter shieldingthat depends on multiple parameter-values.
For instance, suppose SPEC1 is refined as in the following specification SPEC2:
SPEC2: “Charging is allowed only if the Balance is below 190e and Monthly Total
usage is below 390e.”
As the node Charge should be shielded depending on two (i. e., multiple) parametervalues, this is a typical example of the multi-dependent parameter shielding. Note thistime that the test suite of Table 2 is not a valid 2-way test suite anymore, since, e.g., test
case No. 2 is not executable under the refined specification SPEC2. A valid 2-way test
suite under SPEC2 is as shown in Table 3. Further, it is now difficult to model SPEC2
concisely in CTM. The reason is, as explained, a tree node cannot have multiple parents.

Table 3: A valid 2-way test suite under SPEC2, where parameters C.M. and C.A. areshielded when either Balance is >190e or M.T. is >390e.
No. Age Balance C.C. M.T. C.M. C.A.
1 child ≤190e w/o ≤390e cash 50e
2 child ≤190e w/o ≤390e cash 10e
3 child >190e w/o >390e — —4 adult ≤190e with ≤390e cash 10e
5 adult ≤190e with ≤390e c.c. 50e
6 adult >190e with ≤390e — —7 senior ≤190e w/o >390e — —8 senior ≤190e w/o ≤390e c.c. 50e
9 senior ≤190e with ≤390e cash 10e
10 senior >190e with >390e — —charge IC card

C.C.
with

Age

child

Monthly Total

Balancewithout>190€

senior

adult

>390€

≦190€

ChargeAmount

Method

cash

c.c.

―

≦390€

10€

50€

Constraints:・ ￢ (C.C.=with ⋀ Age=child)
・ C.M.=c.c. ⇒ C.C.=with・ (M.T.= >390 ⇒ (C.M. = － ⋀ C.A. = －)) ⋀((C.M. =－ ∨ C.A> =－) ⇒ M.T.= >390)―

Fig. 3: A CTM test model for the IC card system uner SPEC2, which expresses a test
model of the test suite in Table 3.

A possible solution that CTM can do is to explicitly handle the vain values and complex constraints involving them as in Fig. 3. However, we assume such manipulations on
test models makes themselves too complex and busy, losing conciseness, for engineers
to creates and maintain, especially in dealing with industrial-scaled large systems.
In this paper, we propose CTMshield , by extending CTM with parameter shieldingthat can depend on multiple parameter-values, or more generally an arbitrary logic formula. Fig. 4 shows the basic idea of the extension, by showing an test model example
in CTMshield which expresses SPEC2. Observe that CTMshield is extended with the additional description called (parameter) shielding conditions. Observe also that the shielding condition in Fig. 4 specifies SPEC2 directly, using the notation “P ←shield V” tomean parameter-value V shields parameter P. In such a way, we aim to avoid explicitly handling the vain values and complex constraints to express parameter shielding,and thus to retain test models concise and readable. To evaluate the effectiveness of the
proposed extension, we conduct experiments via case studies, where we applied combinatorial testing to test industrial systems in the railway domain, using CTMshield . As
summary, the experimental results showed that parameter shielding was used in 72%
of the cases; CTMshield was able to reduce the tree size by 7.13% and the length of

charge IC card

C.C.

Balancewithout

with

Age

child

>190€

senioradult

Monthly Total

≦190€

≦390€

ChargeMethodcash>390€

Amountc.c. 10€

CONSTRAINTS:・ ￢ (C.C.=with ⋀ Age=child)

・ C.M.=c.c. ⇒ C.C.=with
SHIELDING CONDITIONS:・ Charge ←shield M.T. = >390€)

50€

Fig. 4: A CTMshield test model that expresses SPEC2 and hence is a test model of thetest suite in Table 3.

constraints by 22.9% on average, compared with CTM of [7,8]. Therefore, CTMshieldcontributes to saving human effort on modeling.
The paper is organized as follows. Section 2 mentions related studies. Section 3
clarifies the design of CTMshield . Section 4 reports our experimental study to evaluate
the effectiveness of the proposed extension. Section 5 states our plan for future work.

2

Related work

CTM has been recognized as a key technique in the field of combinatorial testing [2],and has been studied on various aspects. For example, Lehmann and Wegener [5] introduced constraints to the original CTM [7] to extend its expressiveness. Prioritizing test
cases was studied for effective test design in the setting of CTM [6]. A test generation algorithm dedicated to CTM was developed in [8]. Also, experimental data in [8] indicatethat the structured aspect of CTM reduces the lengths of constraints to be described. For
industrial aspects, CTM has been used in industries of safety-critical domains: e. g., it
is used in a standard test documentation in automotive industry [9]. Driven by industrial
demand, tools to support CTM have been developed by several vendors [10].
The notion of parameter shielding in combinatorial testing has been studied in several different approaches. To our knowledge, the earliest work that is relevant for CTM
is by Grochtmann [4]; however, its focus does not seem on parameter shielding but
on diagrammatic approaches, as the phenomenon of parameter shielding was not mentioned. Chen et al. [11] first clarified and defined the notion of parameter shielding inthe setting of Covering Arrays. which considers only unconstrained and unstructured
models. They provided test generation algorithms for this special kind of Covering Arrays. Segall et al. take yet another approach of “common patterns” [12]. They identified
several recurring properties in modeling as patterns, which are often hard to capture correctly, and supply solutions for them. The notion of parameter shielding is captured by
one of their patterns, called “Conditionally-Excluded-Values” pattern. Zhao et al. developed a test generation tool of combinatorial testing, called Cascade, which can handle
shielding parameters explicitly [13] and its handling mechanism is basically same as
the proposed solution in [12].

Our work differs from these works in that our contributions are to propose a modeling language by extending CTM with parameter shielding to advance CTM and evaluate
its effectiveness via case studies.

3

Classification Tree Method with parameter shieldingThis section proposes the modeling language CTMshield , which extends CTM with thenotion of parameter shielding. To be conscious about the extension, we first define thelanguage for combinatorial testing, next that for CTM, and finally that for CTMshield .
The definition of combinatorial testing, whose example is in Fig. 1, is as follows:
Definition 1 (Combinatorial testing). A combinatorial testing model is a tuple m =hP, V, Φi, where P is a set of parameters, V = {V p } p∈P is a family of parameter-values,
where V p is the value domain of p, and constraints Φ are a set of Boolean formula overparameter-values.
A test case is a value assignment to parameters in test model m. Formally, it can be
defined as a function γ : P → V such that γ(p) ∈ V p for every p ∈ P. Note that a test
case γ must satisfy all the constraints Φ (noted as ∀φ ∈ Φ.γ |= φ or γ |= Φ).
A CTM test model consists of a Classification Tree (CT) and constraints. A CT
consists of three kinds of nodes: classifications, which correspond to parameters incombinatorial testing; classes, which correspond to values; and compositions, a notionthat does not appear in combinatorial testing.
Definition 2 (CTM). A test model of CTM is a tuple m = hr, P, V, C, ↑, Φi ∈ M, wherehP, V, Φi forms a test model of combinatorial testing, C is a set of compositions, r ∈ C
is a root node, and ↑ is a function from P ∪ C\{r} to V ∪ C that expresses a part of thechild-parent relation of the tree structure of CT.
As some parameters are shielded and assigned the vain value “—”, a test case of
CTM extends that of combinatorial testing as γ : P → {—} ∪ V, while inheriting γ |= Φ.
A parameter p is shielded, assigned “—”, in a test case γ, if its nearest ancestor value
is not chosen for the parameter in γ. For example, parameter C.M. is shielded in test
case No. 3 in Table 2, since its nearest ancestor value “>190e” is not chosen for theparameter Balance in the test case.
Now, the definition of CTMshield is given as follows:
Definition 3 (CTMshield ). A CTMshield model is a tuple m = hr, P, V, C, ↑, Φ, Φs i ∈ M s ,where hr, P, V, C, ↑, Φi is a CTM model and Φs is a function from P ∪ C to Boolean
formulas. We denote Φs (n) as the (parameter) shielding condition of n ∈ P ∪ C.
The definition of CTMshield extends that of CTM by the shielding conditions Φs . A
test case of CTMshield also inherits that of CTM, including the condition of parametershielding specified by the tree structure. Moreover, in CTMshield a parameter p in a test
case is shielded by Φs when its shielding condition Φs (p) is satisfied by the test case.
In order to express Φs in practice, we take a list of pairs of form p ←shields φindicating that Φs (p) = φ, and assume Φs (q) = False when q is not specified in the list.

Note that a CTM model is a CTMshield model with an empty list of such specifications.
Fig. 4 is an example of CTMshield , and Listing 1.1 shows a formulation of the test modelin Fig. 4 according to Definition 3.
Listing 1.1: A formulation of the test model in Fig. 4 according to Definition 3.
1
2
3
4
5
6
7

8
9

r = { charge IC card (CICC) }C= { Charge }P = { C.C. , Age , Balance , Method , Amount , Monthly Total (M.T.) }VC.C. = { with , without } , VAmount = { 10e , 50e } , VAge = { child , adult , senior ,VBalance = { >190e , ≤190e } , VM.T. = { >390e , ≤390e } ,VMethod = { cash , c.c. } , VC.A. = { 10e , 50e } ,↑ = { (C.C., CICC) , (Age, CICC) , (Balance, CICC) , (Charge, ≤190e) , (Method, Charge), (Amount, Charge) }Φ = { ¬(C.C. = with ∧ Age = child) , C.M. = c.c. ⇒ C.C = with }Φs = { ( >390e, Charge ) }4Case studies and evaluations

This section reports our empirical studies to evaluate the effectiveness of the proposed
extension, i. e., CTMshield extending CTM with parameter shielding, through case studies with industrial systems.
We determine the “effectiveness” of CTMshield by the conciseness of test models
in CTMshield compared to those in CTM (the one dealt in [5,8]). More specifically,
we measure the conciseness of models on their description complexity in terms of thenumber of parameter-values and the length of constraints needed to describe the models.
Our evaluation poses the following three research questions:
RQ1: How often is the parameter shielding condition used? What is its usage rate?
RQ2: How many tree nodes can CTMshield reduce, compared with CTM?
RQ3: How much length of constraints can CTMshield reduce, compared with CTM?
4.1

Setting

In the case studies, we applied combinatorial testing to functional testing of 25 systemlevel functions of the following two distinct industrial systems in railway domain, andin doing so we used CTMshield for modeling the functions: 19 functions from a ticket
gate system (system A) and 7 functions from a payment system (system B).
For comparison between CTMshield and CTM, we also prepared test models in CTM.
We prepare a program to translate test models in CTMshield to equivalent ones in CTM;
i. e., it handles complex manipulation of constraints and dummy nodes. For example, it
inputs the CTMshield test model in Fig. 4, and outputs the CTM test model in Fig. 3.
Next, we measure the following metrics of the test models in CTMshield and CTM:
1. P/V: The size of parameter-values; this is expressed as gk11 gk22 ...gknn , which meansthat for each i there are ki parameters that have gi values, following [14,15].

Table 4: Summary of experimental results.
spec.

P/V

CTM shield
#N
|Φ|

e.g.1

5 1

2

P/V

1

3 2 1

22

3

6

A-2
214 61
58
A-1
215 33 41 51
76
A-3
223 32
93
A-4
219 31 42
103
A-6
218
71
A-5 219 34 52 61 101 112
26 31
27
A-8
A-7 217 33 91 131 151 331 159
A-9
27 51
36
A-10
29 31
39
A-11
210 3
46
A-12
237 71
139
A-13
29 31
31
216 33
67
A-14
A-15
210 31
42
A-16
27 31
25
A-17
23 81
23
23
17
A-18
A-19
215
58

33
32
38
34
38
317
0
0
34
31
31
34
0
0
0
0
0
0
0

9 42
6 41
24 31 41
12 43
24 46
51 44
0 42
0 47
12 31
3 51
3 41
12 43
0 0
0 0
0 0
0 0
0 0
0 0
0 0

B-1
B-2
B-3
B-4
B-5
B-6

2 3

l(Φ) |Φ s | l(Φ s )12 217 42
13 243 46
25 37
26 38
26 37 41211

3

65 12 28 39 42 0 45
165 13 216 327 46 0 417 71
43 22 35 45 18 44
50 23 35 46 27 45
51 23 35 45 51 33 45
33
210 31
0 41

3

CTM
#N

17

8.3

47.1

6.3
1.3
5.1
3.7
10.0
3.4
6.7
4.1
2.6
2.4
2.1
2.1
0
0
0
0
0
0
0
4.2

51.4
23.1
27.9
27.3
25.0
21.2
0
0
0
0
0
0
0
0
0
0
0
0
0
14.7

10.2
12.2
13.6
14.1
13.8
2.0
11.0
7.13

72.2
58.1
41.4
33.8
31.2
0
39.4
22.9

12 28 39 42
13 216 327 46
22 35 45
23 35 46
23 35 45 51
210 31

24

3 11

1

8
210 34 61
62
33 13235
4
214 34 41 51
77
3 3 4113
7
218 3798 38 41 1514312215 35 42
107 34 42 13133
24
210 3879 38 43 131 151 64
16 216 36 41 52 61 101 116 317 42 132 85
8
24 33
29
42
8
28 212 38 91 141 161 331 169
4728
3
26 31 51
37
35
15
5
28 32
40
3 1 5184
29 33
47
3 1 417
12
234 33 71142
3 4 43
24
0
29 31
31
0
0
0
216 3367
0
0
0
210 31
42
0
0
0
27 31
25
0
0
0
23 81
23
0
0
0
23
17
0
0
0
215
58
0
0
avg. of system A
20
75
16
20
20
4

2 3 4

N
l(Φ) ∆%
(%)∆Φ
% (%)

|Φ|
2

87 45 251 271 72
220 415 252 691 179
58 36 42 151 171 58
63 39 43 151 171 71
64 311 43 151 171 77
49
41
4
avg. of system B
total avg.

2. #N: The number of nodes; this is the summation of the numbers of compositions,parameters, and values.
3. l(φ): The length of constraints φ, which is defined in a similar way as [16] as follows: l(a) = 1 for all atoms a, l(¬P) = 1 + l(P), and l(P ∗ Q) = 1 + l(P) + l(Q) where∗ is a binary operator of ∧, ∨, ⇒, and ⇔. E.g., l(¬1 P2 = v1 2 ∨3 P3 = v1 4 ) = 4.
Since CTMshield has an additional description component of parameter shielding
conditions, we also measure the following two metrics for test models of CTMshield :4. l(Φs ): The length of parameter shielding conditions Φs . As mentioned in Section 3
and exemplified in Fig. 4, a shielding condition is expressed using “←shield ” inpractice; e. g., “P1 ←shield ¬P2 = v1 ∨ P3 = v1 ” to mean Φs (P1 ) = ¬P2 = v1 ∨ P3 =v2 ”, where P1 , P2 , P3 expresses parameters and v1 and v2 values. Thus, we define
the length of a shielding condition for parameter n by l(Φs (n)) + 2, regarding ←shield
as a binary operator. For example, l(Φs (P1 )) = l(¬1 P2 = v1 2 ∨3 P3 = v1 4 ) + 2 = 6
5. |Φs |: The size of Φs in the form 1k1 2k2 ...nki , which this time means there are ki
conditions whose length is n for each n ∈ Nat while nki are omitted if ki = 0.
In order to quantitatively answer the research questions, from data about the test
model of CTM (mc ) and that of CTM (m s ) for each function, we retrieve the following:

N– the reduction rate of the number of nodes ∆%
:
N
∆%
=:

#N mc − #N ms
#N mc

(1)

– the reduction rate of constraint length ∆Φ
%:
∆Φ
% =:

s
l(Φmc ) − (l(Φms ) + l(Φms ))l(Φmc )

(2)

where #N m , l(Φm ), and l(Φms ) respectively mean the number of nodes, length of constraints, length of shielding conditions in test model m.
Note that ∆Φ% considers not only the length of constraints, but also the length ofshielding conditions for test model in CTMshield m s . This is for a fair comparison. We
expect (and will see) CTMshield can in fact reduce the length of the constraints. However,this is achieved at the cost of describing the shielding conditions. To avoid such an
unfair comparison, we designed in ∆Φ% to consider not only constraints length but also
the length of shielding conditions in test models of CTMshield .
4.2

Results and observations

Table 4 summarizes the experimental results. The first column shows retrieved datafrom the test model example in CTMshield in Fig. 4 and an equivalent test model in
CTM in Fig. 3, whose main points are read as follows:
1. The CTMshield test model in Fig. 4 has five parameters for two values and one
parameter for three values, hence its P/V is expressed as 25 31 . On the other hand,
the CTM test model in Fig. 3 has three parameters for two values, two for threevalues, and one for four values, hence its P/V is expressed as 23 32 41 .
2. The number of nodes in the CT in CTM is 24, while that in CTMshield is 22. Thus,Nthe number of nodes is reduced by 2 (= 24 − 22), and the node reduction rate (∆%)
2
is 8.3%(= 24 ).
3. The constraint length of the CTM test model is 11, while that of CTMshield is 6. Wealso take the description cost of shielding conditions for CTMshield into account, as
the length of shielding conditions which is 3. Thus, according to the definition, the(17−(3+6))
8
reduction rate of constraint length (∆Φ= 17).
% ) is 47.1%(=
17
From the summary of the experimental results shown in Table 4, we answer the
research questions as follows:– Answer for RQ1: The shielding conditions were not necessarily used for all thecases; instead, they are used in 12 out of 19 cases (63.1%) for system A and in
all the six cases (100%) for system B; hence 72% (= 18/25) in total of systems A
and B.
– Answer for RQ2: For the cases where parameter shielding are used, the reductionNrate of the number of tree nodes by CTMshield (∆%) is on average 4.2% for system
A and 11.0% for system B; 7.13% on the total average.

– Answer for RQ3: For the cases where parameter shielding are used, CTMshield
reduces the constraint length, compared with CTM, (i. e., ∆Φ% ) on average by 14.7%
for system A, by 39.4% for system B; by 22.9% on the total average.
Note that all the test models for the functions in both systems are expressed as trees,from which we can consider structured and diagrammatic modeling approach of CTM isuseful and effective in practice. Also, CTMshield shows a higher effectiveness in system
B than system A, from which we may consider that the effectiveness of using CTMshielddiffers between systems. As shorter and simpler constraints reduce the human effort onmodeling, we consider CTMshield to be effective in real-world settings.

5

Conclusion and future work

This work tackled a modeling problem in combinatorial testing, which is a main concern
for its use in real developments. We extend CTM, which have been studied and used asa practical modeling language in combinatorial testing, with parameter shielding, andproposed CTMshield . Our experiments via case studies confirmed its effectiveness.
We plan to conduct more empirical studies to evaluate the effectiveness of CTMshieldwhen used to model industrial systems. We leave to future work a theoretical analysisof the proposed extension, such as consistency arguments with different formalisms forparameter shielding [11], theoretical analysis of effectiveness of the extension such as
the maximum reduction of the number of nodes, the size of constraints per shielding
condition, etc. We also plan to extend our combinatorial testing tool Calot [8,17,18]
with this feature of parameter shielding.
Acknowledgement This work is partly supported by JST A-STEP grant AS2524001H.

References
1. Kuhn, D.R., Wallace, D.R., Gallo, A.M.: Software fault interactions and implications forsoftware testing. IEEE Trans. Software Eng. 30(6) (2004) 418–421
2. Kuhn, R.D., Kacker, R.N., Lei, Y.: Introduction to Combinatorial Testing. CRC Press (2013)
3. Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press (2008)
4. Grochtmann, M.: Test case design using classification trees. In: Proc. of STAR 1994. (1994)
5. Lehmann, E., Wegener, J.: Test case design by means of the CTE XL. In: Proc. of EuroSTAR.
(2000)
6. Kruse, P.M., Luniak, M.: Automated test case generation using Classification Trees. Software Quality Professional (2010) 4–12
7. Grochtmann, M., Wegener, J.: Test case design using Classification Trees and theClassification-Tree Editor CTE. In: Proc. of QW’95. (1995)
8. Kitamura, T., Yamada, A., Hatayama, G., Artho, C., Choi, E.H., Do, N.T.B., Oiwa, Y.,
Sakuragi, S.: Combinatorial testing for tree-structured test models. In: Proc. of QRS 2015,
IEEE CPS (2015) 141–150
9. MODISTARC: OSEK/VDX operating system test plan version 2.0 http://www.osekvdx.org/ (1999) Accessed: 2016-03-07.
10. Berner & Mattner: TESTONA ver.5.1.2 http://www.testona.net/en.

11. Chen, B., Yan, J., Zhang, J.: Combinatorial testing with shielding parameters. In: Proc. of
APSEC 2010, IEEE CPS (2010) 280–289
12. Segall, I., Tzoref-Brill, R., Zlotnick, A.: Common patterns in combinatorial models. In:
Proc. of ICST 2012, Montreal, 2012, IEEE CPS (2012) 624–629
13. Zhao, Y., Zhang, Z., Yan, J., Zhang, J.: CASCADE: A test generation tool for combinatorial
testing. In: Proc. of ICSTW 2013, IEEE CPS (2013) 267–270
14. Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: An approach totesting based on combinatiorial design. IEEE Trans. Software Eng. 23(7) (1997) 437–444
15. Garvin, B.J., Cohen, M.B., Dwyer, M.B.: An improved meta-heuristic search for constrainedinteraction testing. In: Proc. of SSBSE 2009. (2009) 13–22
16. Büning, H.K., Lettmann, T.: Propositional Logic: Deduction and Algorithms. Cambridge
University Press (1999)
17. Yamada, A., Kitamura, T., Artho, C., Choi, E.H., Oiwa, Y., Biere, A.: Optimization of combinatorial testing by incremental SAT solving. In: Proc. of ICST 2015, IEEE CPS (2015)
1–10
18. Yamada, A., Biere, A., Artho, C., Kitamura, T., Choi, E.: Greedy combinatorial test casegeneration using unsatisfiable cores. In: Proc. of ASE 2016, IEEE CPS (2016) 614–624