Assessment and Etiology of Attention Deficit Hyperactivity Disorder and Oppositional Defiant Disorder in Boys and Girls

Attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) are more common in boys than girls. In this paper, we investigated whether the prevalence differences are attributable to measurement bias. In addition, we examined sex differences in the genetic and environmental influences on variation in these behaviors. Teachers completed the Conners Teacher Rating Scale-Revised:Short version (CTRS-R:S) in a sample of 800 male and 851 female 7-year-old Dutch twins. No sex differences in the factor structure of the CTRS-R:S were found, implying the absence of measurement bias. The heritabilities for both ADHD and ODD were high and were the same in boys and girls. However, partly different genes are expressed in boys and girls.
In both clinical and population samples, children diagnosed with attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) are predominantly male (Gaub and Carlson 1997; Biederman et al. 2002; Loeber et al. 2000). These sex differences could either be the result of a higher liability for these disorders in boys than girls, or could be attributable to a sex effect in the actual measurement of the phenotype. For example, boys and girls with similar levels of problem behavior may receive systematically different rating scores if the items of the instrument do not reflect the problem behaviors in the same manner in boys and girls. This may conceptualized as bias with respect to sex. In this study, we will investigate this issue in the measurement of ADHD and ODD by means of the Conners Teacher Rating Scale-Revised:Short (CTRS-R:S) version.
The presence of sex differences in the prevalence of ADHD and ODD also raises the question whether there are sex differences in the etiology of these disorders. Sex differences in etiology can only be interpreted if the measurement of a disorder is not biased with respect to sex. Lubke et al. (2004) discussed in detail the importance of establishing unbiasedness with respect to sex for the correct interpretation of any sex differences in the results of genetic modeling. The aim of this paper is, therefore, to investigate measurement bias in teacher ratings of ADHD and ODD with respect to sex. If unbiasedness can be shown to be tenable to reasonable approximation, we shall, as the second goal of this paper, estimate the genetic and environmental contributions to the phenotypic variance in ADHD and ODD in boys and girls.
Mellenbergh (1989) defined unbiasedness, or equivalently, measurement invariance (MI), with respect to group to mean that the distribution of the observed test score, conditional on the latent construct that the test measures, is identical over groups (e.g., boys and girls). In more simple terms, this means that the instrument is measuring the same construct in boys and girls (Mellenbergh 1989; Meredith 1993). If this is the case, we expect the score of a given person to depend on that person’s score on the latent construct, but not on that person’s sex. If MI does not hold, a boy and a girl, who are characterized by the same degree of problem behavior, may obtain systematically (i.e., regardless of measurement error) different scores on the instrument. This is undesirable, because we wish our measurements to reflect accurate and interpretable differences between cases in different groups.
Analyses aimed explicitly at establishing MI with respect to sex, according to the approach outlined by Meredith (1993), have yet to be conducted with respect to ADHD and ODD. Although MI has not been investigated, there have been some studies, which addressed sex differences in the factor structure in teacher ratings of ADHD. Fantuzzo et al. (2001) examined the factor structure of the 28-item version of the CTRS with exploratory factor analyses. They reported a three-factor solution, which accounted for 58% of the variance. The factors admitted interpretations in terms of conduct, hyperactivity, and passivity. The invariance of the factor structure was established by comparing results from random subgroups with the results from subgroups based on sex. The results supported the similarity of the factor structure across sex. A concern in this study is that the subjects in this study were 580 children from 33 classrooms located in low-income neighborhoods. It is, therefore, unclear how representative the sample is of the general population. Furthermore, because each teacher rated more than one child, and children were clustered into classes, the assumption of independent observations, which is important in statistical inference, might not hold.
In this study, we conducted confirmatory factor analyses (CFA) of data from a large general population sample of 7-year-old twins, rated by their teachers on ADHD and ODD. Two questions are addressed. First is the measurement model that relates differences in the latent constructs of ADHD and ODD to the observed behavior problem scores identical in boys and girls, i.e., is MI tenable? Second, do the magnitudes of the genetic and environmental influences differ, or do different genes play a role in boys and girls?
Methods
Subjects and procedure
This study is part of an ongoing twin study of development and psychopathology in the Netherlands. The subjects were all registered at the Netherlands Twin Registry (NTR; Boomsma et al. 2002, 2006). We assessed a sample of Dutch twins from the birth cohorts 1992–1996. These twins were assessed by their teachers when they were 7 years old.
The twins at age 3 are representative of Dutch 3-year-old children with respect to their scores on measures such as the Child Behavior Checklist (CBCL; van den Oord et al. 1995). The socioeconomic status of the parents of the twins was somewhat higher than the level in the general Dutch population (Rietveld et al. 2004). When twins reached the age of 7 years, parents were asked to provide informed consent to approach the teacher. Consent was given by 80.1% of the parents. Teachers of these pairs received a questionnaire by mail, and were asked to return it to the NTR by mail. The response rate of the teachers was 80.0%. CTRS data were available for at least one twin in 1,651 twin-pairs (1,511 complete pairs).
The maternal CBCL-AP scores at age 7 years were not significantly different between families in which parents provided permission to approach the teachers (mean = 2.95, SD = 2.93) and families in which parents did not do so (mean = 3.15, SD = 3.18; t(3,063) = 2.0, p = 0.133). However, mean maternal AP ratings were significantly higher in twins whose teachers did not return the questionnaire (mean = 3.34; SD = 3.13) than in twins whose teachers did return the questionnaire (mean = 2.78; SD = 2.81; F(1) = 16.82, p < 0.001).
To avoid biased test results due to statistical dependency of the twin data, we randomly included the score from only one of the members of a twin-pair in the CFA. The resulting sample for CFA consists of 1,651 individual twins (800 boys and 851 girls). In the genetic analyses, we included all complete twin-pairs. Data were available for both members of a twin-pair in 248 MZ male, 251 DZ male, 294 MZ female, 234 DZ female, and 484 DZ opposite sex pairs. Some twins were rated by the same teacher (877 pairs, 58%) while the remaining twins were rated by different teachers (634 pairs, 42%). The genetic analyses accounted for potential differences between same and different teacher ratings by using the model developed by Simonoff et al. (1998). Incomplete twin-pairs were excluded from the genetic analyses.
Measure
The CTRS-R is a widely used instrument to assess behavior problems (Conners 2001; Conners et al. 1998). The CTRS-R was developed by factor analyzing a large set of items, and including items that load highly on interpretable common factors. In addition to the scales that were derived based on factor analysis, an ADHD index was included. This index comprises the best 12 items for distinguishing children with ADHD from children without ADHD as assessed by the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association 1994; Conners 2001). The short version of the CTRS-R, which was used in this study, contains 28 items from the following scales: oppositional (ODD, five items); cognitive problems-inattention (IN, five items); hyperactivity (HI, seven items); and the ADHD index (ADHD-I, 12 items). One of the items (item 27; excitable, impulsive) is included in both the hyperactivity scale and the ADHD index. The items are rated on a 4-point Likert scale for symptom severity (i.e., 0 = ”not true at all”; 1 = ”just a little true”; 2 = ”pretty much true”; 3 = ”very much true”). The internal consistency and stability of the CTRS-R:S version are good, as the Cronbach’s alpha coefficients range from 0.88 to 0.95, and 6–8-week test–retest correlations range from 0.72 to 0.92 (Conners 2001).
Distribution of the items
Because of the categorical nature of the item scores, Pearson product moment correlations underestimate the correlation of the underlying latent trait, and consequently the parameter estimates obtained in factor analyses or principal component analyses based on the Pearson correlation matrices are biased (Dolan 1994). We, therefore, adopted an approach that is suitable for factor analyzing discrete item scores. Polychoric correlations between items were obtained based on the liability threshold model (Lynch and Walsh 1998). In the case of a 4-point Likert scale, three thresholds divide the latent liability distribution into four categories.
Criteria of MI
The criteria of MI are empirically testable in the common factor model (Meredith 1993). MI criteria are: (1) equality of factor loadings over groups; (2) equality of item intercepts over groups (i.e., differences in item means are the result of differences in factor means), and (3) equality of residual variances (i.e., variance in the observed variables, not explained by the common factor) over groups. When satisfied, these restrictions ensure that any differences in the mean and variance of the observed variables are due to differences in the mean and variance of the common factor.
For ordinal data, MI can be tested by constraining the thresholds and factor loadings, and residual variances to be equal in boys and girls. These constraints allow estimation of group differences in the means and variances of the common factor. To this end, the mean and variance of the common factor are fixed at 0 and 1, respectively, in an arbitrary reference group. We chose to estimate the mean and variance in girls, and to use boys as the reference group. In doing so, we are modeling the observed group differences as a function of the differences in the latent liability.
Statistical analyses
The polychoric correlation matrices of the items of the four subscales were calculated using Prelis (Jöreskog and Sörbom 1996). All CFA were performed on raw data using Mx (Neale et al. 2003). In principle, factor analysis of p ordinal items can be carried out using full information maximum likelihood (FIML) estimation. However, this requires repeated integration of the p-variate normal distribution, which can become computationally demanding even with as few as 12 items. We, therefore, estimated the model parameters using marginal maximum likelihood estimation (MML; Bock and Aitkin 1981). MML maximizes the likelihood of the data conditional on the latent trait, in contrast to FIML, which maximizes the unconditional likelihood. The advantage of MML compared to FIML is that it is computationally much less demanding.
To test if the Conners scales are MI with respect to sex, the factor structure was constrained to be identical in boys and girls. The ODD, IN, and HI scales resulted from factor analyses, and a single factor was fit to these scales. In contrast, the ADHD-I contains items related to problems with inattention and hyperactivity, and thus a two-factor model was fitted. To detect prevalence differences in ADHD and ODD across sex, the means and variances of the latent factors were constrained to be equal in boys and girls.
The fit of the models was compared by χ² tests, with a type I error probability set at α = 0.01. Browne et al. (2002) noted a complication of the χ² test. Specifically, they showed that χ² is influenced by the unique variances of the items. If a trait is measured reliably, the inter-correlations of the items are high, the unique variances are small, and the χ² test may suggest a poor fit even when the differences between the expected and observed covariance matrices are trivial. The standardized root mean square residual (SRMR; Bentler 1995) is a fit index that is not sensitive to the size of the correlations. To avoid the rejection of a simpler model due to high inter-item correlations, we only reject a model if a significant χ² test is accompanied by large residuals (SRMR > 0.08; Hu and Bentler 1999).
After investigating MI with respect to sex, we look at sex differences in the genetic and environmental influences on the individual differences in the sum scores of the scales, given that MI is tenable (Lubke et al. 2004). The polychoric correlations of the four scales were calculated by sex and zygosity in PRELIS (Jöreskog and Sörbom 1996). The genetic analyses were performed on the raw data using Mx (Neale et al. 2003).
With data from MZ and DZ twins, the variance in behavior can be attributed to genetic and environmental factors. In our sample, 58% of the twins were in the same classroom and 42% of the twins were in different classrooms. Correlations between twins may be higher when children are rated by the same teacher. Therefore, a correlated error model developed by Simonoff et al. (1998) was used to analyze the data. In this model, individual differences in behavior are explained by additive genetic factors (A), common environmental factors that are shared between two twins of a pair (C), and unique environmental factors (E).
The unique environmental factors are allowed to correlate in twins who are placed into the same classroom, and do not correlate in twins who are placed into different classrooms. For the genetic analyses, the items of each subscale were summed, and the sum-score was recoded so that three thresholds divide the latent liability distribution into four categories. The thresholds were chosen in such a way that the categories contain more or less equal numbers of subjects. We preferred this procedure to the analysis of the raw sum scores, because these are skewed, and therefore cannot be analyzed with maximum likelihood approaches based on the assumption of normality (Derks et al. 2004).
Sex differences in genetic and environmental influences were examined in two ways. First, we investigated if the estimates of the genetic and environmental variances are equal in boys and girls. Secondly, we investigated if the same genes influence phenotypic variation in boys and girls. These qualitative sex differences were evaluated by constraining the genetic correlation in opposite sex twins at 0.5 (similar as in same-sex DZ twins). If different genes play a role in boys and girls, the genetic correlation is expected to be lower than 0.5 in opposite sex twins.
Results
Measurement invariance
We tested for MI by constraining the factor loadings, thresholds, and residual covariance matrices to be equal for boys and girls while allowing the factor means and variances to be different. The factor structure of ODD was MI with respect to sex (χ²(18) = 16.66, p > 0.10; SRMR = 0.01 and 0.06 in boys and girls, respectively). MI was also tenable for the ADHD-I (χ²(55) = 70.41, p > 0.05; SRMR = .03 and 0.05 in boys and girls, respectively). Both IN and HI showed statistically significant different factor structures in boys and girls (χ²(18) = 98.45, p < .001, and χ²(26) = 57.99, p < 0.001, respectively). However, the residuals between expected correlation matrices under the constrained and the non-constrained model were small (SRMR = 0.01 in girls and SRMR = 0.02 in boys, for both IN and HI). Apparently, the lack of fit was the result of the high inter-correlations between the items (Browne et al. 2002), and not of large residuals between the expected covariance matrices. Therefore, we tentatively accept MI with respect to sex. Table 1 provides the factor loadings and thresholds of the best-fitting models. We also included the factor loadings as reported by Conners (2001) to facilitate the comparison of our sample with the sample that was used to create the scales. Note that the factor loadings, as reported by Conners (2001), are generally much lower, as these estimates are based on the assumption of a continuous, normal distribution of the item scores, an assumption that is obviously violated in the instance of a four-category Likert scale.
Table 1Promax rotated factor loadings and thresholds (T) of the best-fitting factor model for the four subscales of the CTRS-R:SItemsLoadings factor 1Loadings factor 2Loadings as reported by Conners (2001) (N = 1,897)T1T2T3Oppositional2. Defiant0.91—0.480.711.652.456. Defies0.88—0.501.022.002.7410. Spiteful0.69—0.301.492.323.1315. Argues0.84—0.510.821.802.5220. Explosive0.78—0.411.201.923.12Cognitive problems-Inattention4. Forgets things0.93—0.550.341.291.828. Poor spelling0.81—0.500.270.951.4413. Poor reading0.71—0.450.390.871.2118. Lacks interest0.70—0.471.011.652.2722. Poor arithmetic0.82—0.490.551.221.68Hyperactivity3. Restless0.73—0.620.100.941.517. Always on the go0.79—0.570.661.311.8711. Leaves seat0.82—0.580.621.371.8317. Difficulty waiting0.83—0.650.120.911.5821. Runs about0.79—0.441.211.892.5124. Difficulty playing0.87—0.520.341.172.0327. Excitable0.86—0.660.401.121.84ADHD indexAP1. Inattentive0.99−0.09—−0.360.761.4714. Attention span0.95−0.03—0.030.871.5516. Only pays attention0.450.23—0.361.201.8819. Distractibility0.94−0.01—0.591.542.2525. Fails to finish0.720.09—0.241.271.9926. Not follow instructions0.680.09—0.381.301.86HI5. Disturbs0.050.80—0.201.031.529. Remain still0.050.88—0.921.682.2412. Fidgets0.160.67—0.120.941.5323. Interrupts−0.050.85—0.541.372.0727. Excitable−0.090.91—0.461.151.8328. Restless−0.060.97—0.501.191.81AP attention problems; HI hyperactivityThe thresholds are constrained to be equal in boys and girls. The means and variances of the latent factors in boys are constrained at 0 and 1, respectively. The means and variances of the latent factors are freely estimated in girls. The variances in girls were not significantly different from 1. The means of the latent factors are estimated at −0.60 (oppositional), −0.86 (hyperactivity), −0.42 (ADHD-I AP), and −0.52 (ADHD-I-HI). The mean of the cognitive problems-inattention factor is not significantly different between boys and girls
Genetic analyses
Having established MI of the CTRS-R:S scales with respect to sex, we estimated the twin-correlations and carried out a genetic analysis of the data. Twin correlations are shown in Table 2, for same and different teachers, respectively. The genetic model fitting results of the four scales are reported in Table 3. All correlations are higher in MZ twins than in DZ twins, suggesting the presence of genetic influences on individual differences. The correlations are higher in twin pairs rated by the same teacher than for twin pairs rated by different teachers. This was taken into account by using a correlated error model (Simonoff et al. 1998). The lower correlations in opposite-sex DZ twins than in same-sex DZ twins suggest that different genes play a role in boys and girls.
Table 2Polychoric twin-correlations of the CTRS rated by same teachers (ST) versus different teachers (DT)OppositionalCognitive problems-inattentionHyperactivityADHD indexSTDTSTDTSTDTSTDTMonozygotic males0.860.530.900.760.810.590.850.59Dizygotic males0.500.490.640.290.420.140.490.19Monozygotic females0.870.430.920.640.830.470.870.52Dizygotic females0.660.100.600.420.340.250.450.24Opposite-sex twins0.370.130.440.230.300.170.370.27Table 3Genetic model fitting results on CPRS-R:S ratingsModelParameters-2 LLWith modelΔdfΔχ²pOppositional behavior in 7-year-old children1. Fully saturated844779.36————2. Equal correlations same and different teachers784812.261632.91<0.0013. ACE model, estimates of ACE are allowed to be different between same and different teachers444817.9414038.580.5344. ACE, ACE same teacher = ACE different teacher404853.843435.90<0.0015. Correlated errors ACE model424819.8014240.440.5406. Correlated errors AE model404823.93524.130.1277. Correlated errors AE model, AE boys = AE girls384829.76625.830.0548. Correlated errors AE model, AE boys = AE girls, opposite-sex genetic correlation-free374821.86717.900.005Cognitive problems-inattention in 7-year-old children1. Fully saturated846785.90————2. Equal correlations same and different teachers786822.571636.67<0.0013. ACE model, estimates of ACE are allowed to be different between same and different teachers446815.9414030.040.8744. ACE, ACE same teacher = ACE different teacher406843.873427.93<0.0015. Correlated errors ACE model426815.8514229.950.9186. Correlated errors AE model406820.01524.160.1257. Correlated errors AE model, AE boys = AE girls386821.99621.980.3728. Correlated errors AE model, AE boys = AE girls, opposite-sex genetic correlation-free396817.50714.490.034Hyperactivity in 7-year-old children1. Fully saturated846880.07————2. Equal correlations same and different teachers786906.971626.90<0.0013. ACE model, estimates of ACE are allowed to be different between same and different teachers446914.5914034.420.7154. ACE, ACE same teacher = ACE different teacher406942.353427.76<0.0015. Correlated errors ACE model426924.1214244.050.3856. Correlated errors AE model406926.76522.640.2677. Correlated errors AE model, AE boys = AE girls386929.06622.300.3178. Correlated errors AE model, AE boys = AE girls, opposite sex genetic correlation-free376919.99719.070.003ADHD in 7-year-old children1. Fully saturated847420.21————2. Equal correlations same and different teachers787458.101637.89<0.0013. ACE model, estimates of ACE are allowed to be different between same and different teachers447472.5914052.380.0914. ACE, ACE same teacher = ACE different teacher407507.403434.81<0.0015. Correlated errors ACE model427477.7814257.570.0556. Correlated errors AE model407479.14521.360.5077. Correlated errors AE model, AE boys = AE girls387480.72621.580.4548. Correlated errors AE model, AE boys = AE girls, opposite-sex genetic correlation-free377475.93714.790.029
Model fitting analyses showed that variation in all four scales could be explained by additive genetic and unique environmental effects. The influences of the shared environment were not statistically significantly. The magnitude of the influences of genes and environment did not differ between boys and girls. The standardized estimates of genetic and environmental influences are shown in Table 4. Genetic effects explained 56–71% of the variation in the CTRS subscales. Unique environmental effects explained the remaining 29–44% of the variation. For all four scales, the genetic correlation was significantly lower than 0.5 in opposite-sex twins. This implies that different genes are expressed in males and females. The genetic correlation in opposite-sex twins was 0.16 for oppositional behavior, 0.35 for cognitive problems-inattention, 0.21 for hyperactivity-impulsivity, and 0.32 for the ADHD index.
Table 4Standardized estimates of the genetic and environmental effects on problem behavior% Additive genetic effects (A)% Non-shared environmental effects (E)Genetic correlation opposite-sex twinsOppositional56440.16Cognitive problems-inattention71290.35Hyperactivity58420.21ADHD index61390.32
Discussion
The purpose of this study was twofold. First, we investigated if teacher ratings on ADHD and ODD are measurement invariant with respect to sex. Secondly, genetic and environmental influences on variation in ADHD and ODD were compared between boys and girls.
Measurement invariance
Teacher ratings on ADHD and ODD were found to be measurement invariant with respect to sex. In other words, teacher assessments of these behavior problems relate to the same latent variables in boys and girls. Sex differences in observed scores on ADHD and ODD can, therefore, be interpreted as differences with respect to the latent construct. This supports the contention that the reported sex differences in ADHD and ODD (Gaub and Carlson 1997; Loeber et al. 2000; Maughan et al. 2004) are due to a higher liability for the disorder in boys than girls and not to measurement bias.
Quantitative and qualitative differences in the heritability among boys and girls
More than half the variance in ADHD and ODD in boys and girls is attributable to genetic influences. The remaining variance is attributable to unique environmental influences. The magnitude of the influences of genes and environment is the same in boys and girls. However, part of the variance in ADHD and ODD is attributable to different genes in boys and girls. We base this on the fact that the genetic correlation between DZ opposite-sex twins was significantly lower than 0.5, which is the theoretical value (in the absence of assortative mating), if the same genes influence behavior in boys and girls. We observed a genetic correlation lower than 0.5 in DZ opposite-sex twins for oppositional behavior, cognitive problems-inattention, hyperactivity, and the ADHD index.
Few studies have addressed quantitative and qualitative sex differences in heritability estimates from teacher ratings. Saudino et al. (2005) reported qualitative sex differences in heritability of teacher ratings of hyperactive behavior in 7-year-old twins. They did not report any quantitative sex differences, which is in agreement with the current findings. Vierikko et al. (2004) report lower correlations in opposite-sex twins than in same-sex DZ twins of teacher ratings of hyperactivity-impulsivity in 12-year-old twins. However, both genetic and shared environmental effects were found to contribute to the phenotypic variance in these data. It was not possible to determine if the lower opposite-sex correlations were the result of sex-specific genetic influences or sex-specific shared environmental influences, although the presence of the latter appeared more likely. These findings disagree with the current results in the sense that we did not find any evidence for shared environmental influences. However, both studies suggest that teacher ratings are influenced by partly different etiological factors in boys and girls.
The finding of different genetic influences in boys and girls in teacher ratings stands in contrast with results based on parental reports. In parent ratings, qualitative sex differences are not found for attention problems (Rietveld et al. 2004) or ODD (Hudziak et al. 2005). The different findings in parent and teacher ratings may be explained by the fact that the behavior of children depends on the context in which they are observed. Apparently, inattentive, hyperactive, and oppositional behavior of boys and girls are influenced by partly non-overlapping factors at school, while this is not true for these behaviors at home.
The finding of sex-specific genetic variation has implications for gene-finding studies of ADHD and ODD. The fact that the genes which influence the behavior of boys and girls do not completely overlap indicates that some quantitative trait loci may explain variation in boys but not in girls and vice versa. Therefore, the data from boys and girls cannot be collapsed when studying genetic effects in teacher ratings.
In the NTR, teacher data are collected at the ages 7, 10, and 12 years. The sample sizes at the ages 10 and 12 are currently relatively small. In the future, we plan to address the issue of qualitative sex differences in teacher ratings in a longitudinal framework. The results of such a study will reveal if the finding of sex differences in the specific genes that play a role is also present in older children. Another important issue that may be addressed is the MI of ADHD with respect to age.
The results of this study should be interpreted in the light of the following limitations. First, we did not replicate the factor structure of the CTRS-R:S by means of exploratory factor analyses of the 28 items. To take the ordinal nature of the data into account, we used the liability threshold model (Lynch and Walsh 1998). We limited the number of common factors to keep the computational burden manageable. Therefore, we performed CFA, in which we assumed that the items are correctly assigned to the four scales and that cross-loadings are absent. Second, teacher ratings were shown to be measurement invariant with respect to sex, but this finding may not generalize to parent ratings. The correlations between Conners parent and teacher ratings are small to moderate with a range 0.18–0.52 (Conners 2001). It has been shown that parents and teachers rate partly different aspects of the child’s behavior (Derks et al. in press; Martin et al. 2002). Future studies will reveal if MI is also tenable in parent ratings.
Assessment of ADHD and ODD symptoms, through teacher reports on the CTRS-R:S, provides a solid starting point for measuring sex differences in mean scores or in heritabilities. Variation in teacher ratings of children’s problem behavior is mainly influenced by genetic factors. The size of the genetic influences does not depend on the child’s sex, but partly different genes are expressed in boys and girls. Future studies should reveal if these findings generalize to children from different age groups.

adhd
odd
sex differences
measurement invariance
conners teacher rating scale


A web server for transcription factor binding site prediction

Promoter prediction has gained increased attention in studies related to transcriptional regulation of gene expression.We developed a web server named PMSearch (Poly Matrix Search) which utilizes Position Frequency Matrices (PFMs) to predict transcription factor binding sites (TFBSs) in DNA sequences. PMSearch takes PFMs (either user-defined or retrieved from local dataset which currently contains 507 PFMs from Transfac Public 7.0 and JASPAR) and DNA sequences of interest as the input, then scans the DNA sequences with PFMs and reports the sites of high scores as the putative binding sites. The output of the server includes 1) A plot for the distribution of predicted TFBS along the DNA sequence, 2) A table listing location, score and motif for each putative binding site, and 3) Clusters of predicted binding sites. PMSearch also provides links for accessing clusters of PFMs that are similar to the input PFMs to facilitate complicated promoter analysis.
Background
Transcription factors play a pivotal role in the regulation of gene expression by sequence specific binding to the promoters of target genes.
Prediction of putative transcription factor binding sites has become an important resource to explore genome organization and regulatory mechanisms.
The binding specificity of transcription factors are usually represented by known sequence motifs (consensus sequences) or matrices (PFM (Position
Frequency Matrix) or PWM (Position Weight Matrix)). [1] High throughput analyses using SELEX (Systematic
Evolution of Ligands by Exponential Enrichment) and CHIP-Chip (Chromatin Immunoprecipitation-Microarray) along with computational sampling methods have
generated thousands of PFMs. These data together with the related transcription factor information are curated in online databases such as Transfac [2], JASPAR [3], etc.
Online applications, such as MatInspector [4,5],
MATCH [6] and ConSite [7], have been built to utilize
PFMs to predict transcription factor binding sites (TFBSs) embedded in promoter sequences. Many of the servers are comprehensive but lack the information on transcription
factors whose binding specificities are similar to the input PFMs, which could effectively assist regulatory module finding. To provide such information and to meet the
needs for efficient prediction tasks in the study of gene regulation network, we developed a server named PMSearch (Poly Matrix Search) for predicting TFBSs in DNA sequences
with the novel functions of fetching similar PFMs and processing multi-forms of input motifs. PMSearch made the following improvements: 1) It has a more succinct and friendly
user-interface. 2) It provides user with the convenience of customizing any set of PFMs from the local dataset. 3) It generates a resizable plot that shows the distribution
and scores of predicted binding sites in variable scales. 4) It provides clusters of PFMs similar to the input PFMs.
Methodology
We have implemented a scoring scheme adapted from
previous algorithms. [4,6] The uninformative nucleotides (ambiguous letters: N, B,
D, H, and V) on either end of an input PFM are discarded before searching to enhance efficiency and specificity. We downloaded 507 PFMs from JASPAR and Transfac Public 7.0 to construct
a local dataset from which the user may take any subset of PFMs, or along with userdefined PFMs, for a prediction task. In addition to PFMs, the server could also accept sequence motifs
(modeled in the format of consensus sequences), which will then be converted into pseudo PFMs and applied in subsequent prediction. We have set a default global cutoff value of 0.85 previously
adopted by TFSEARCH. [9] The user may adjust this cutoff if required.
User interface
The web interface of PMSearch helps in promoter analysis. The inputs are composed of PFMs and sequences under analysis. The user may submit PFMs, motifs or retrieve desired
PFMs from our dataset using keywords (e.g., name of the transcription factor), its source database accession number (e.g. Transfac Matrix Accession number M00001) or our local
accession number (e.g. X00001). Sequences in FASTA format or Genbank format are acceptable and will be auto-parsed. When the user initiates a predication task, each sequence will
fork an independent task in which corresponding results can be retrieved respectively. In the output, PMSearch plots a figure that illustrates binding sites that are scored above
the cutoff value (Figure 1). The user may specify a sub-region on the sequence that will be plotted for more detailed view, such as a specific promoter region in a sequence file.
Other results including scores, sequence motifs and closely located putative binding sites (clusters) are also listed. In addition, the server provides a hyperlink for the user to
fetch PFMs that are related to the input PFMs by implementing a gap-allowed alignment algorithm. A comprehensive instruction for users is available online.
Conclusion
PMSearch is an easy-to-use and efficient tool that utilizes PFMs to predict transcription factor binding sites in DNA sequences. It offers user the flexibility to search for
putative TFBS with any set of PFMs. PMSearch outputs a plot demonstrating distribution of predicted binding sites and a table of the locations, scores, motifs and clusters of predicted binding
sites. In addition, PMSearch provides the user access to PFMs that are relative to an input PFM for more sophisticated promoter analysis, as the predicted binding sites of one transcription
factor could also be bond with other transcription factors that share the similar binding specificity. Such information may give hints to untangle the composite transcription regulatory network.
We propose to update PFMs in our local dataset regularly. The source codes are available from the authors upon request.
Caveats
It has been proposed that many of the predicted TFBSs lack biological function in vivo. [10] This could result from 1) The
predicted site is located in a context which is insufficient to facilitate transcription factor binding, 2) The low specificity of the input PFM gives rise to large portion of false
positive predictions. We suggest the users prepare a list of certain transcription factors that are suspected to regulate the target gene to specify a prediction task.

web server
transcription factor binding site
motif
position frequency matrix


Mastering DICOM with DVTk

The Digital Imaging and Communications in Medicine (DICOM) Validation Toolkit (DVTk) is an open-source framework with potential value for anyone working with the DICOM standard. DICOM’s flexibility requires hands-on experience in understanding ways in which the standard’s interpretation may vary among vendors. DVTk was developed as a clinical engineering tool to aid and accelerate DICOM integration at clinical sites. DVTk is used to provide an independent measurement of the accuracy of a product’s DICOM interface, according to both the DICOM standard and the product’s conformance statement. DVTk has stand-alone tools and a framework with which developers can create new tools. We provide an overview of the architecture of the toolkit, sample scenarios of its utility, and evidence of its relative ease of use. Our goal is to encourage involvement in this open-source project and attract developers to build off and further enrich this platform for DICOM integration testing.
INTRODUCTION
Digital Imaging and Communications in Medicine (DICOM)1 plays a major role in the health care information technology (IT) field as the standard for medical images and communication throughout the hospital. With the organization of the DICOM Standards Committee in 1996 and the support of major medical groups and imaging vendors worldwide, DICOM has become a dominant integration mechanism in the hospital enterprise. The DICOM standard version 3.0 contains 16 parts (more than 2,000 pages), and the standard itself is constantly evolving as new software and imaging technologies are developed. Mastering a technology like DICOM can be a daunting task, but the successful student will be on the right path with the DICOM Validation Toolkit (DVTk) from DVTk.org. Consider DVTk the lab kit that is handed out with the DICOM standard on the first day of DICOM 101 class.
As with any other technology, true learning and understanding require hands-on experience.2 Although the standard itself does not define or identify testing or validation procedures to assess conformance, a number of third-party tools have been developed to fill that role. DVTk is a powerful tool for mastering the intricate DICOM file format and transfer syntax. The mantra of DVTk is to make DICOM easy. If a picture archive and communication system (PACS) adminsitrator, for example, is having a problem with imaging device integration, DVTk can be the first line of defense in tracking the problem. When vendors are pointing fingers at one another, DVTk can help them past recriminations and on to real solutions. For medical software developers, integrators, and testers, DVTk can help to more quickly produce robust systems.
The DICOM Validation Tool (DVT) test framework is a flexible architecture and uses service–object pair (SOP) class definition files, making it adaptable as the DICOM standard evolves. It includes a graphical user interface (GUI) and a command line interface, DICOM media validation, service class user (SCU) and service class provider (SCP) emulators, and a rich scripting language. For more advanced testing, VBScript (Microsoft Visual Basic Scripting edition; Microsoft; Redmond, WA) or JScript (Microsoft) can be executed by DVT, and a set of .NET assemblies is available for developing stand-alone test tools with languages such as Visual Basic.NET (Microsoft) and C# (Microsoft).
If a validation tool is to be taken seriously, it must be vendor neutral. With the release of DVT 2.1 in 2005, DVTk now exists as an open-source community project. Not only does this encourage more vendors to contribute (because of the incentive of reduced development and integration costs), but the adoption and proliferation of standards make it easier for individual developers to contribute to open-source projects such as DVTk. Independent software developers are sometimes discouraged from proceeding because of the chance that their work will provide a “one-off” solution only and not be widely used. DVTk, with the backing of the mature DICOM standard, is attracting talented developers with a desire to create something useful and lasting. This success assures that time devoted to learning this toolkit will not be wasted.
This article focuses primarily on the DVT main application. We provide background on the toolkit and specific examples about the two intended roles of DVT: service and development. Current efforts within the DVTk project and future directions are also highlighted.
DVTk HISTORY AND ARCHITECTURE
In 2000, Agfa HealthCare (Mortsel, Belgium) and Philips Medical Systems (Eindhoven, The Netherlands) decided to coordinate activities around DICOM validation testing by bringing together efforts already started by both companies under a joint DVT project. The intention was to produce a DVT that could not only be used internally by both companies to test their own products but also made freely available to other original equipment manufacturers (OEMs) as a means of testing their products to the same level of detail. The ultimate aim was to reduce the time spent integrating proprietary systems by first exposing these systems’ equipment to tests run using DVT.
DVT project has a steering committee with responsibility for guiding legal, technical, and commercial aspects. The steering committee meets every 6 months to discuss past progress, current issues, and future requirements. A project manager was elected by the steering committee to manage the DVT project on a daily basis and report back to the committee. Development tasks were divided up based on the available skills of developers who report to the project manager. In its first years, Agfa and Philips provided the personnel to staff the DVT project.
In September 2005, DVT was made into an open-source project (http://www.dvtk.org, DICOM Validation Toolkit; last accessed May 2007) under SourceForge (http://sourceforge.net/projects/dvt, SOURCEFORGE.NET ; last accessed May 2007) as DVTk and is licensed under the GNU Lesser General Public License (LGPL). The steering committee decided that the time had come to begin promoting DVTk to a wider audience, with the aim of attracting other companies who might join and supply development resources. Around this time, ICT Healthcare (Eindhoven, The Netherlands), which had already been supplying development resources to the project, joined as a full member with representation on the steering committee. The goal was to make DVTk the independent gold standard for DICOM validation and thereby improve the interoperability of all vendor products using the DICOM interface.
The dvtk.org Web site is now the location for the latest downloads, defect tracking details, and forums on using DVTk. Interested individuals and companies may join the project through the Web site. A weekly project telephone conference coordinates the activities of the development team.
DVTk-based applications include DVT, the main application, which together with the core forms what is now referred to as the DICOM Validation Framework (Fig. 1). The core includes a DICOM testing data model and object-oriented class structure. The DVTk and DVTkData libraries provide access to this data model and class structure via the managed code adapter. The core includes the DICOMScript language that is supported by DVT. This language can be separated into basic programming, which includes the SEND and RECEIVE commands for simulating DICOM SCPs and SCUs, and advanced programming. The advanced programming language includes commands such as SYSTEM, for executing operating system native applications, and a number of commands for working with the Data Warehouse feature. The Data Warehouse is a run-time memory structure in which the user can store Association Control Service Element (ACSE) requests and responses, DICOM commands, and DICOM objects for reuse across test scripts and sessions.
Fig 1.DICOM Validation Framework.
For more advanced testing scenarios, DVTk includes the Script Support library and the High-Level Interface (HLI) library. The HLI library is a newer abstraction built on top of the core that makes it easy to write multithreaded tests. The application program interface (API) exposed by this library encapsulates many of the low-level core API classes and methods that make writing VBScripts similar to writing scripts in DVTk’s native DICOMScript language. VBScripts can be executed entirely within the DVT GUI application or command-line executable or debugged in Visual Studio .NET. Other DVTk-based applications built on the core and currently available on dvtk.org include: DICOM Network Analyzer, a network sniffer and DICOM protocol analyzer; DICOM Editor, for displaying and editing DICOM files; DICOM Compare Tool, for comparing the attributes and values of two DICOM files; DICOM Attribute Validator, for validating DICOM files, including Structured Report objects, against definition files; DICOM File Stripper, for removing all but mandatory attributes from a DICOM file; and DICOM File Anonymizer, for removing patient and physician information from a DICOM file.
To date, the following contributions have been made to the DVTk project by outside parties, companies, and/or institutions:
Medical Communications (UK) provided the underlying Transmission Control Protocol/Internet Protocol (TCP/IP) Capture File to the DICOM protocol data unit (PDU) conversion utilities used by the DICOM Network Analyzer application.The National Institute of Standards and Technology (NIST; Gaithersburg, MD) provided the Health Level Seven (HL7)3 validation Web services used in the latest DVTk HL7 validation components.
GETTING STARTED WITH DVTk OUT OF THE BOX
The latest version of DVT can be downloaded from http://www.dvtk.org . It comes packaged as a Microsoft Windows InstallShield application. The installation subfolder includes a user’s guide and Windows help files for the extensible .NET assemblies (available only if MS Visual Studio .NET is installed). A large number of examples on how to use the many features of DVT are included in the subfolder.
Starting DVT presents the user with an empty workspace. DVT is designed to work on a single project at a time, although multiple views of the project may be opened from which multiple tests may be simultaneously executed. A DVT project is a container for one or more test sessions. A session is a container for the configuration of one or more tests to be performed against a system under test (SUT). Project and session configuration properties are stored in flat files. The DVT GUI exposes most of the configuration properties, although some of the more advanced settings, such as STRICT-VALIDATION, are available only by directly editing the session file. Some settings can also be modified in script files. For example, the STRICT-VALIDATION script command overrides the value of the STRICT-VALIDATION session property. Some settings, such as CALLED_AE_TITLE, also have built-in default values that are assumed if the property is not defined in a script or session file.
Session files come in three types: emulator, script, and media. Emulator sessions are used when DVT should act as an SCU or SCP emulator to test the DICOM transfer syntax. The emulators have support for Verification SCU/SCP, Storage SCU/SCP, and Print SCP. Script sessions are used when DVT is used to execute a DICOMScript, DICOMSuperScript, or VBScript. A DICOMSuperScript is simply a script that calls one or more DICOMScripts. Script sessions have support for network SCP/SCU message exchange and DICOM media file creation. Media sessions are used to validate the DICOM file format as a DICOMDIR and/or DCM media files. When validating a DICOMDIR file, any referenced files are also validated.
An easy way to gain familiarity with DVT is go through the examples in the subfolder using DVT as both the test tool and the SUT. Opening up the example project displays a tree view of all test sessions defined in the project and provides details on the session currently selected in the tree. Clicking on the Window menu and selecting the New Project View and Tile item displays two views of the project.
The next example will walk through the Modality Worklist (WLM) SCP/SCU test script sessions. Selecting the WLM_SCP session in the top view and the WLM_SCU session in the bottom view presents a view from which two test sessions may be executed simultaneously (Fig. 2).
Fig 2.DVT-Worklist Management SCP and SCU test sessions.
At the top of the Session Information tab are general settings, including the session type, session ID (used in uniquely naming results files), and location of files used/created by the test session. The DVT Roles Settings section defines the Application Entity (AE) title and some connection settings that DVT assumes during the test session. The SUT Settings section defines the AE title and some connection settings, including the TCP/IP address, of the system the session is testing. Because this example uses DVT as both the testing system and the SUT, the DVT role settings in the top view are synchronized to the SUT settings in the bottom view, and the SUT settings in the top view are synchronized to the DVT role settings in the bottom view.
Locations of the SOP class definition files required for the test session are selected under the Specify SOP Classes tab. The selected definition files are loaded into memory when the test session starts, and DVT uses these to validate the DICOM messages and objects it sends and receives. A definition file describes a single DICOM SOP class in terms of the combination of DICOM Message Service Element (DIMSE) commands and Information Object Definitions (IODs) that make up the SOP class. In this case, both test sessions specify the Modality Worklist Information Model–FIND SOP Class item. The standard definition files that come with DVT in the definitions subfolder are taken directly from the DICOM standard parts 3 and 4. Private definition files can be made that extend the validation capabilities of DVT by copying one of these standard files and modifying it with private user IDs (UIDs), modules, and attributes. A typical customization might specify that when a particular SUT is known to always send a value for the Patient Name (0010,0010) attribute, the corresponding definition file can be modified to define the Patient Name attribute as a type 1 (mandatory, nonzero-length) attribute.
Opening the WLM_SCP tree node in the top view and selecting the DICOMScript, 1.ds, displays a Script tab in the right side of the view. This script defines the test steps for the SCP. Simple DICOMScripts such as this can be written using only the SEND and RECEIVE commands. A large number of additional commands are available for more advanced testing scenarios. The Script tab is read-only, but Windows Notepad can be launched from a context menu on the script name in the tree view. A handy DVT DICOM script reference (dvtDICOMscript.hlp) file is in the docs folder. This script first executes a RECEIVE command for an association request message that specifies a single presentation context, consisting of the Modality Worklist Information Model–FIND SOP Class and three possible transfer syntaxes. The SCU script in the bottom view SENDs an association request message. The only required parameter for the ASSOCIATE-RQ message is the presentation context. For the SCP’s RECEIVE command, the ASSOCIATE-RQ message is referred to as the reference or expected object. This will be compared to the received ASSOCIATE-RQ message. DVT will perform validation in two steps:
The received object is first validated against any loaded definition files. This step ensures that the correct attributes are present in the object and that they are encoded correctly.This step is optional and applied only if a reference object is present in the script. Checks made here are that the expected number of attribute values has been received and that each attribute value matches the corresponding reference object attribute value.
Often, it is not known with what values an SUT will respond, in which case no attributes can be specified in the reference object, so that only step one validation occurs (http://www.dvtk.org “DVT User Guide”, version 2.1, August 2005). The VALIDATION script command determines how the DICOM objects are validated by DVT. By default, validation is ENABLED, meaning full validation will occur. The value of the STRICT-VALIDATION session property determines how the result of the validation is handled by DVT. If STRICT-VALIDATION is enabled, the presence of attributes in the received message must match the definitions exactly. If they do not match, then DVT reports a FAILED validation and aborts further DICOMScript interpretation. If STRICT-VALIDATION is disabled and attributes do not match, then DVT reports a WARN message. By default, STRICT-VALIDATION is disabled.
Following a successful association validation and negotiation, the SCU script SENDs a C-FIND-RQ message of type Modality Worklist–FIND. The SCP script RECEIVEs the C-FIND-RQ, validates it, and proceeds to SEND three hard-coded C-FIND-RSP messages. The SCU script was written to mirror the SCP script, so the three reference C-FIND-RSP messages it defines match exactly with those sent by the SCP. These scripts also demonstrate two DVT scripting features: Value Mapping and Value Representation (VR) Keywords. Value mapping allows the user to substitute a label for a value defined in a script, generated by DVT, or received from the SUT, and then refer to that value with the label further down in the script. The LABEL: keyword is used to map an attribute value defined in the script or received from the SUT. The NEW: keyword tells DVT to generate a new value and assign it to the given label. An example of this is in the SCP script, where the Study Instance UID attribute (0020,000D) is assigned a new value generated by DVT and named with the label StudyInstanceUid1. The generated UID value can be referred to subsequently in the script as StudyInstanceUid1 (although this was not done in this example). VR keywords, such as the AUTOSET keyword used in both the SCU and SCP scripts, tell DVT to generate a value and assign it to the corresponding attribute. In the example case, AUTOSET is used in the Scheduled Procedure Step Start Date attribute to ensure that values match between the two scripts. Both Value Mapping and VR Keywords are sensitive to the type of attribute to which they are applied.
This test is begun by right clicking on the SCP script and selecting execute. When DVT is executing a test session, all tabs except the Activity Logging tab are hidden, the session tree in the corresponding view is disabled, and the test stop button in the toolbar is enabled. The SCP’s Activity Log will indicate that it is waiting for a connection on port 104. Executing the SCU script in the lower window will result in a number of messages in the Activity Logs, followed by the termination of the test sessions. Comparing and correlating the SCP and SCU activity log entries provides a good visual reference for the DICOM message flow between two AEs.
By default, the AUTO-TYPE-2-ATTRIBUTES session property is set to true in the session file, which means that DVT will automatically add any zero-length type 2 (type 2 attributes must be included; however, they may be encoded with a zero-length value or no value) attributes from the definition file to the dataset before sending to the SUT. This behavior is recorded in the SCU’s activity column by the “Automatic Type 2 Attributes population...” statement. This feature means that only type 1 (required, nonzero-length) attributes must be explicitly stated in the scripts.
After completion of the scripts, each view displays the Validation Results tab, where the results of the test sessions are displayed. DVT stores the results of each test in a _res.xml file in the test session’s configured results directory. The file name takes the form <Detail∣Summary>_nnn_<scriptName>_res.xml, where nnn is the session ID from the session properties. Varying the session ID from one test execution to the next allows storage of multiple sets of results in the results directory. Two session Boolean properties define whether summary or detailed results are generated: SUMMARY-VALIDATION-RESULTS and DETAILED-VALIDATION-RESULTS. If the test session’s STORAGE_MODE property is set to as-media or as-dataset, any media files received by DVT during the test are also stored in the results directory, again using the session ID to help uniquely name the files. The generated results files are listed in the session tree under the script name. Selecting one of the results files displays it in the Validation Results tab. The Validation Results tab is a Hypertext Markup Language viewer allowing navigation between the summary results, detailed results, and any generated media files using hypertext links. DVT does not include a DICOM file viewer. To automatically view a generated .dcm file by clicking on a link in the Validation Results tab, a DICOM image viewer must be installed and associated with the .dcm file type.
The SCP’s Validation Results tab will show a test result of PASSED, and a .dcm media file is created that contains the Modality Worklist–FIND dataset received in the C-FIND-RQ message sent from the SCU. The SCU’s Validation Results tab also shows a result of FAILED, with nine errors reported. The Summary Results File lists the errors; in this case, three type 1 attributes are missing from each C-FIND-RSP message sent from the SCP. Clicking on a Link to Detailed Result link displays the errors in the context of the complete C-FIND-RSP message. The detailed results file contains the complete contents of each message sent and received in chronological order. Any comment lines in the DICOMScript that begin with ## are copied directly to the detailed results file. This is a good way to insert additional test information directly into the results. For additional troubleshooting output in the Activity Logging tab and detailed results file, the LOG-RELATION, LOG-DEBUG, LOG-DULP-STATE, and PDU-DUMP test session properties can be enabled. In the example described here, it is left as an exercise for the user to add the missing type 1 attributes to the SCP script so that the SCU test session result becomes PASSED.
DICOMSuperScripts (script files with a .dss extension) enable the reuse of DICOMScripts in various test scenarios. The storage example script sessions included with DVT demonstrate the benefits of DICOMSuperScripts.
DVTk FOR SERVICE TROUBLESHOOTING
As demonstrated in the previous section, DVTk can be a useful tool when troubleshooting modality worklist problems. Another common use case would occur when adding new modalities to a PACS network. It can be frustrating to ensure that AE titles, ports, and IP addresses match between the modality and PACS configuration. This section will illustrate ways in which DVT can be used to troubleshoot a PACS-modality interface problem and describe another, more service-orientated tool available from DVTk.org.
Because DVT can act as both an SCP and SCU, it is an excellent starting point for troubleshooting PACS-modality problems. One of the most common and most frustrating failures is when a modality does not send images to the PACS. Most modalities provide little or no error information when image storage problems occur with the PACS. The PACS system is often equally unhelpful. Without an error message of some kind, PACS administrators have no guidance on where to begin in addressing the problem. DVT can simulate the modality or the PACS to pinpoint the source of the problem while obtaining hard evidence that can be used to engage the involvement of additional service layers, such as the hospital networking group or modality vendor. The Emulator_1 test session in the example project that comes with DVT installation can be used to emulate the problem modality. The problem modality’s AE title and PACS connection properties must be copied to the Emulator_1 session’s properties. Although not absolutely necessary, the modality being emulated should be taken off the network to prevent any AE title conflicts during troubleshooting. The first test to perform is a simple Verification to ensure that there is DICOM network communication between the modality AE and the PACS. The test is started by opening the Emulator_1 session’s tree node, right clicking on the Storage SCU Emulator node, and selecting the Execute menu item. All tabs except the Activity Logging tab are hidden, and a dialog box is displayed. Clicking on the Echo button tells DVT to send a C-ECHO message to the SUT. The activity logging tab will display the operations performed. For the purposes of this article, the tests are run against the open-source DICOM PACS DCM4CHEE (http://www.dcm4che.org, dcm4chee-2.x DICOM Clinical Data Manager system; last accessed May 2007).
If the verification test succeeds, then one can assume the patency of DICOM network connectivity between the modality and PACS. The next test is to attempt image storage to the PACS. This test requires a set of DICOM test images, preferably from the modality that is experiencing the problem. The images should contain no names, IDs, or UIDs that will conflict with real-world data. It is important, however, that the data used to perform the test matches the real-world data in structure as closely as possible, including any private data elements the modality creates. A copy of the modality’s DICOM conformance statement comes in handy here. This will help simulate the behavior of the modality as closely as possible. Under the Specify Transfer Syntax (TS) button on the Session Information tab, all transfer syntaxes supported by the modality as a storage SCU can be selected. For example, to test a computed tomography (CT) modality, one would select the CT Image Storage SOP Class (1.2.840.10008.5.1.4.1.1.2). For this test, the storage SOP class and transfer syntax that match the DICOM test images that will be stored to the PACS should be selected. Note that this step is not actually necessary; when emulating a storage SCU, DVT will automatically add to the association negotiation the transfer syntaxes and SOP Classes from the DICOM media files being stored. Selecting the correct transfer syntaxes and SOP Classes is necessary when emulating a storage SCP, however. Executing the Storage SCU Emulator opens the Storage SCU Emulator dialog box. In the dialog box, the Add button is used to add the test images to the list that will be sent to the SCP. Selecting the Validate before export option tells DVT to validate the media files against the corresponding definition file prior to sending them. The number of associations the modality uses when sending multiple images is also specified here (most modalities would send multiple images on a single association). Finally, clicking the Send button will tell DVT to execute the storage test and close the dialog. DVT’s detailed results will show the verification of each media file followed by each storage transaction. The Storage SCU emulator does not automatically do storage commitment as the SCP emulator does. To simulate the modality performing a storage commitment request, one could create a DICOMScript session similar to the Commit_SCU example session, modifying the SOP instance UIDs to match the images stored. This test should be performed immediately after the storage test.
To demonstrate some of the more advanced DICOMScript features and the Data Warehouse in DVT, one can imagine a scenario where a CT modality has recently undergone a software upgrade. Since the upgrade, technicians have been reporting that although they can successfully postprocess images on the modality before sending them to the PACS, processing of images retrieved from the PACS back to the modality fails. No errors have been reported by the PACS or modality on storage or retrieval. One possible reason is that the PACS is doing something to the stored images that is preventing the modality from processing them. To evaluate, one can perform what is called a PACS transparency test. The idea is to make what the PACS is doing (if anything) to the images transparent by comparing the before and after images. This test will require a DICOM CT image file from the modality on the DVT workstation and a DVT project with two sessions. A script session playing the role of SCU is used to put the image in the DVT Data Warehouse, store it to the PACS, and initiate a C-MOVE from the PACS back to DVT. An Emulator SCP session is used to receive the image moved back from the PACS and for performing the validation of the received image against the original image in the Data Warehouse.
The first commands the script performs are to reset the Data Warehouse and read the test image into the Data Warehouse (Fig. 3). The first READ command loads the DICOM image into the Data Warehouse and uses the value of attribute 0×00080018, the SOP Instance UID, as a reference value. This will allow the SCP Emulator that subsequently receives the image back from the PACS to automatically compare it to the image in the Data Warehouse based on the fact that the SOP Instance UID values are the same. The second READ command loads the same image again into the Data Warehouse but this time references it with CTIMAGE1. This reference will be used to export the image directly from the Data Warehouse to the PACS. As an alternative to READing the image into the Data Warehouse, one could use the CREATE and SET commands to build an image object directly in the Data Warehouse. DVT has the ability to generate pixel data patterns for VRs of type other byte (OB), other float (OF), and other word (OW). For the purposes of this example, it is best to use an image from the problem CT modality. The next operation the script performs is to establish a C-STORE CT Image association with the PACS. After establishing the association, the script creates a C-STORE-RQ command object in the Data Warehouse and exports the CT Image referenced by CTIMAGE1 using the C-STORE-RQ command referenced by CSTOREREQID, over the established association (Fig. 4). After closing the storage association, the script creates a new association on which to perform the C-MOVE, sends the request to move the image from the PACS to the DVT AE, and then closes the association (Fig. 5). Note that the Patient ID (0×00100020) and SOP Instance UID (0×00080018) were manually copied from the CT Image test file. Also, before executing this test, the SCP Emulator must already be started and waiting for the C-STORE from the PACS in response to the C-MOVE request. If the test executes successfully, the statement “Reference Dataset with identifier ‘1.3.46.670589.10.900123.19970114.35042000040’ found in Warehouse” should appear in the Activity Log indicating that DVT found the same CT image previously loaded into the Data Warehouse and will use it to perform a comparison of the dataset attributes and values.
Fig 3.PACS transparency test using DICOMScript-reading image into the Data Warehouse.Fig 4.PACS transparency test using DICOMScript-exporting image from the Data Warehouse.Fig 5.PACS transparency test using DICOMScript-moving the image back to DVT.
This test requires that the PatientRootQueryRetrieve-MOVE.def definition file is loaded in the script session and the CTImageStorage.def definition file is loaded in the Emulator SCP session. If, as suspected, the PACS is modifying the CT Image object before returning it to the modality, the Emulator SCP results data will identify where these modifications occurred. In this case, the hypothetical tester notices that DVT is indicating that a set of private attributes in the Data Warehouse copy of the image are missing from the image returned from the PACS. Upon further investigation, it turns out that the software upgrade performed on the modality included the addition of a set of new private attributes required by the modality to perform new postprocessing algorithms on the images. The PACS does not support these new private attributes for some reason and is stripping them from the image objects.
Instead of using a combination of a DICOMScript test session and Emulator test session, this test could have been implemented with a single VBScript session using the HLI library. In this case, separate threads would be created for the SCU and the SCP parts. This approach is left up to the user as an exercise.
Although DVT can analyze all aspects of DICOM interfacing, it tends to be overloaded for field service issues. DVTk.org has a number of service-orientated tools also based on the DVTk framework. One of these is the DICOMSniffer & Analyzer Tool. The tool is a GUI-based network sniffer and DICOM protocol analyzer. This tool uses the WinPcap open-source network library for packet capturing and filtering and includes a user guide. This tool can sniff a live DICOM network stream given two endpoints, such as a modality and PACS. The troubleshooting power of this tool should not be underestimated. It can perform message validation between two devices similar to DVT but does not require replacing an SUT device with the testing tool. Figure 6 shows the tool capturing a modality worklist query/response communication between two IP addresses.
Fig 6.DICOM Sniffer & Analyzer tool in capturing mode.
After the capture process, the user is presented with a per-association analysis of the DICOM control and datasets that were captured. The Association Overview tab shows the requested and accepted services and each field of the association PDUs. In the Service Elements Overview tab (Fig. 7), one can see all of the DIMSE messages that were transmitted during the association, save a copy, and view the PDUs. The summary and detailed validation results are similar to those produced by DVT, including message validation against SOP Class definition files. The tool also has the option of saving the captured data to a capture file for latter analysis or for sending on to additional service personnel. The DICOM Sniffer & Analyzer is a tool every PACS administrator should have on his or her workstation.
Fig 7.DICOM Sniffer & Analyzer tool in analyzing mode.
Two more tools built on the DVTk framework that should be in the imaging professional’s toolbox in working with DICOM on a regular basis are DICOM Compare and DICOM Editor. DICOM Compare can compare the attributes and values of two DICOM files. It can also filter out of the compare process any attributes and sequences to make it easier to identify offending elements. A known “good” DICOM object can be compared to one causing a problem that was captured from the network with DICOM Sniffer.
With the DICOM Editor, one can add/delete/modify any attribute or sequence and sequence item and save the modified DICOM file to any location. This tool goes hand-in-hand with the DVT, DICOM Sniffer, and DICOM Compare tools. After those tools have identified the DICOM elements that may be causing a DICOM object storage problem, the editor can be used to manually fix the DICOM object and then resend it to the system where the failure is occurring. If the store succeeds, chances are the source of the problem has been identified.
DVT AS A DEVELOPMENT TEST TOOL
Another use of DVT is automated unit or system testing in a software development environment. Most modern source code management systems include automated build-and-test subsystems. Scripts can be created to test the DICOM interfaces of a vendor’s product, and the command line version of DVT can then be launched to execute those scripts against the product as part of the normal automated build-and-test processes. The DVT results and output files can then be saved along with the rest of the build/test artifacts. The command line version of DVT can be called on a DICOMScript as follows: dvtcmd Modality_System_Test.ses Modality_System_Test.ds. The summary and detailed results Extensible Markup Language files output by DVT make it easy for an automated test system to determine the results of the test, generate a report, and take any other appropriate action, such as e-mailing the development team.
A tool as programmable and rich in output as DVT is valuable to the entire iterative development process. Software validation teams can link test session files, scripts, and result files to software defect issues, so that a developer can then use them to reproduce the defect. A validator can subsequently use those same scripts to test the fixed software. Maintaining a repository of DVT scripts is also an excellent way to run regression tests.
If the DICOM software being tested is in the .NET language family, tighter testing integration with DVTk can be achieved via NUnit,4 the open-source unit testing framework for .NET. This integration uses a DVTkInvoker class to invoke the command line version of DVT with a session file, script file, and results file as arguments. One can use DVT Clients and Servers within NUnit to handle the various DICOM SOP Classes, starting and stopping DVT as necessary and obtaining validation results information. One future goal is to be able to start up and test an entire workflow scenario from NUnit using DVT.
DVTk makes it quick and easy to perform repeatable, detailed testing of DICOM interfaces. In the following example, we posit a stress test to run against a storage SCP service. Because this is a stress test, the test application must hit the SCP simultaneously from multiple SCUs, so that multiple threads will be needed. Writing multithreaded Visual Basic (VB) test applications is a breeze with the new HLI library. In fact, it looks quite similar to a DICOMScript, with high-level abstractions for creating and releasing associations and for sending and receiving messages. This example using the HLI requires DVTk alpha version 2.1.007.006 or greater , which can be downloaded from dvtk.org after registering as a Plus User. Plus Users have access to early releases and draft documents, such as the draft version of the HLI API help file.
In Visual Studio .NET, a VB console project is created and references to the Dvtk.dll, DvtkData.dll, and DvtkHighLevelInterface.dll libraries are added. This simple stress test application takes three arguments: the number of modalities to create (each modality runs on a separate thread), the number of images each modality will store to the SCP under test, and the location of the DVTk definition files. Figure 8 shows the imports required and the Sub Main(ByVal CmdArgs() As String) routine in module Module1. The first operation this application entry point method performs is to call Dvtk.Setup.Initialize(). This must be performed before any calls to the DVTk libraries, matched by the last call in the application, which should be Dvtk.Setup.Terminate(). Then it creates and initializes the MainDicomThread object, attaches it to the activity logging form, starts the main DICOM thread, and finally waits for the threads to complete. After the threads are complete, it logs selected performance data gathered during the stress test.
Fig 8.Multithreaded stress test using VB.NET-main subroutine and imports.
Figure 9 shows the MainDicomThread class that inherits from the DvtkHighLevelInterface.Dicom.Threads.DicomThread class. A single instance of this class is started from the entry point method. This class’s thread execute method starts all of the modalities and returns. As it creates each modality (instances of CtStorageScu), it generates unique identifying information for each SCU, such as AE title. Some of the options are analogous to test session properties. This test application could have loaded a script session configuration file and initialized some of the options from it, but in this case, all of the necessary session options are set directly in the code. After an SCU is created, it is immediately started by calling its thread Start() method.
Fig 9.Multithreaded stress test using VB.NET-MainDicomThread class.
Figure 10 shows a CtStorageScu class that also inherits from DvtkHighLevelInterface.DICOM.Threads.DICOMThread. The purpose of the execute method is to C-STORE a specified number of images to the SCP under test. It first creates unique patient, study, and series level identifiers for the images that will be stored. It then creates the C-STORE CT Image association; then loops to store the images, cloning a CT image dataset that was read from a DICOM CT image file in the constructor; creates unique image identifiers for each image; sends the C-STORE-RQ message; receives the response message; and checks the response status value. Finally, it logs its total storage time for later analysis and closes the association. This application could have been created and executed from the DVT GUI, but developing it in Visual Studio allows the application to be debugged and compiled to an executable file. This application is executed from the command line as: Storage_SCP_Stress_Test.exe 20 1000 “C:\Program Files\DVT\Definitions”.
Fig 10.Multithreaded stress test using VB.NET-CtStorageScu class.
Figure 11 is an excerpt from the Hl1Form that receives all of the logging from the DicomThread objects. Notice the multithreading capabilities of the HLI in this output. While modality 10 is validating an A-ASSOCIATE-AC message it received, modality 3 is handling a CSTORERSP message. This test created 20 CT modalities that are each simultaneously sending a 1,000-image series to the PACS.
Fig 11.Multithreaded stress test-activity logging.
RECENT EXTENSIONS AND FUTURE WORK
DVTk has been extended recently to support HL7 validation in addition to DICOM validation. This is currently used by the DVTk Integrating the Healthcare Enterprise (IHE)5 Actors framework where it is possible to configure DVT to emulate the role of certain actors in the IHE integration profiles. The idea is that the IHE Actors necessary to allow the SUT to be tested are emulated by DVTk. DVTk will then validate the DICOM and HL7 transactions taking place between actors and, in addition, compare the values of certain attributes such as Patient ID, Patient Name, etc., between messages to ensure consistent use.
In the future, the aim is to enhance the capabilities of the DVTk IHE Actors framework by supporting additional functionality in the existing actors and supporting other actors needed in the various IHE Integration Profiles. Support for other protocols is also envisioned.
The DVTk development team is also involved in the new IHE Connectathon Toolkit (code-named “Gazelle”), which is being developed as a successor to the MESA Toolset. It is hoped that the DVTk DICOM validation engine can be wrapped as a Web service for use in Gazelle.
Various new tools will be developed using the DVTk frameworks. A GUI for the DVT-IHE framework; various new emulators, such as a radiology information system emulator and modality emulator; and stand-alone validation applications are likely candidates. The current number of development resources limits what can be done—anyone wishing to contribute can do so via the Web site.
The aim for future IHE extensions is to integrate any other validation services into the DVTk framework as Web services. This may include Cross-Enterprise Document Sharing validation, for example. It is hoped that the Gazelle cooperation will result in web services that can be reused for this purpose.
CONCLUSION
Originally developed by Agfa and Philips to test their products, DVTk has grown into a professionally managed, open-source, vendor neutral, DICOM Validation Framework. Flexible for the changing standard, programmable, and extensible, DVTk is a powerful tool for anyone working with the DICOM standard. The toolkit includes GUI and command line versions of the main validation application, DVT, and a collection of .NET libraries for creating new validation and test tools. The large collection of example validation sessions that come with the toolkit are a great place to start for understanding how the DICOM standard works in practice.
Hospital IT staff can use DVTk for simple tasks, such as pinging the network for the existence of modality AE titles, or for more detailed troubleshooting, such as checking for the presence of specific DICOM attributes and values in messages and media files. Other DVTk-based tools are available for tasks such as editing or comparing DICOM files or for capturing and analyzing messages on a live DICOM stream. The XML-structured test results and media files created by the validation framework provide great evidence for IT staff when discussing a problem with vendors. Developers and testers can create scripts for testing their products DICOM conformance, or build on the framework by creating new standalone test tools. NUnit integration and the command line version of DVT make it possible to incorporate DVTk and the generated test results into automated build-and-test systems.
Current and future efforts include support for IHE actor validation, including HL7 validation, and new device emulators, such as a Radiology Information System (RIS) emulator. As the digital hospital enterprise continues to grow, DVTk is well positioned to take on these new validation roles.
We encourage the reader to continue working with DVTk as a means to master the changing DICOM environment. With the move to an open-source community approach, DVTk is well on its way to becoming the independent gold standard for DICOM interface testing.

dicom
ihe
systems integration
pacs dicom ihe conformance
health level 7


Genetic algorithm for multi-objective experimental optimization

A new software tool making use of a genetic algorithm for multi-objective experimental optimization (GAME.opt) was developed based on a strength Pareto evolutionary algorithm. The software deals with high dimensional variable spaces and unknown interactions of design variables. This approach was evaluated by means of multi-objective test problems replacing the experimental results. A default parameter setting is proposed enabling users without expert knowledge to minimize the experimental effort (small population sizes and few generations).
Introduction
In many technical problems scientists face the problem of identifying optimal process conditions like pH, temperature, concentrations or other variables. A typical example is fermentation medium development. The composition of a fermentation medium consisting of carbon sources, nitrogen sources, mineral salts, trace elements, amino acids and/or peptides, vitamins and other growth factors determines the chemical and nutritional environment of cells in a bioreactor and is thus vital for the effective manufacturing of bioproducts.
Due to the large number of process variables and the metabolic complexity of microorganisms or cells methods of experimental design need to be applied in order to identify values of the relevant variables resulting in an improved performance. Considering several objectives to be optimal there is per definition not one optimal solution but a set of efficient solutions. Mathematically such a multi-objective optimization problem (MOP) can be described in terms of Eq. 1.
The functional relation between the m design variables and the objective functions φ does not necessarily need to be known, as y will be obtained from n experimental observations.
An important criteria for MOPs is that of Pareto optimality, shortly saying that a member of the efficient set is not dominated by any other. In Fig. 1 the principle of dominance is exemplified in the objective space, where for the case of maximizing two objectives the efficient solutions are indicated.
Fig. 1Efficient points (black) are not dominated by any other point. For a dominated point (white) there exist at least one other point that has greater values in both objectives y1 and y2
Classical statistical experimental design methods (Plackett–Burman Design, Response Surface Method) have drawbacks like screening for principal components and assuming an unimodal objective function, reviewed in [1].
More sophisticated stochastic search strategies like genetic algorithms (GAs) have grown in popularity since Rechenberg [2] and Holland [3] first published their work on this subject. GAs are based on evolutionary principles, encoding several sets of design variables on binary strings (individuals in a population) which are processed by GA operators (crossover and mutation) throughout several generations. The principle “survival of the fittest” assures a convergence towards optimal values in the design variables with proceeding generations.
The first GA dealing with multiple objectives was the Vector Evaluated Genetic Algorithm (VEGA) proposed by Schaffer [4]. This multi-objective optimization strategy has already been applied successfully for experimental medium optimization in many cases [1]. The most recent published multi-objective GAs are the Non-Dominated Sorting Genetic Algorithm-II [5] and the Strength Pareto Evolutionary Algorithm (SPEA) [6]. The SPEAs main feature is processing two populations: besides a normal population an external population serves as a kind of archive, keeping track of the efficient set. Especially in the case of experimental design with a limited number of experiments the SPEA is supposed to have advantages compared to VEGA with respect to the experimental effort.
The new software tool with a genetic algorithm for multi-objective experimental optimization making use of SPEA will be outlined. The performance of GAME.opt will be evaluated with the help of special test-functions to achieve appropriate parameter settings for experimental design.
Methods
GAME.opt was developed with LabVIEW 7 (National Instruments). The Application Builder was used to create a stand alone application running under Windows platforms. Data handling is based on spreadsheet files, which can be accessed via a text editor or Microsoft Excel. GAME.opt will be made available on request by the Technical University of Munich.
As the core algorithm is based on the SPEA, fitness assignment, selection and clustering is described in detail by Zitzler and Thiele [6]. To abstract the algorithm: fitness assignment is either based on the strength of an individuals dominance or on the degree an individual is dominated by others. A mating pool equal to the population size is filled by selecting individuals using binary tournament without replacement [7]. Crossover is performed between two parental binary strings from the mating pool at the specified number of crossover points in order to create two off-springs. Each bit of these new strings is inverted with a probability which is defined by the mutation rate. Clustering is used in order to bound the external population to a maximal value Nextern. Each time the external population exceeds this value it is pruned to a size of Ncluster by means of average clustering.
The first step when using GAME.opt is to create a new project where all information on the experimental design is defined. For this purpose the user is led through the steps explained together with some recommendations in the following.
Design variables
The design variables considered in the problem (x in Eq. 1) need to be defined initially. Alternatively to choosing the amount of bits encoding a variable GAME.opt gives the opportunity to choose the amount of levels, resulting from the variable’s bounds and its increment. The choice of levels can be necessary for practical purposes but causes problems in the coding of a decision variable as follows. A general decoding function linearly transforms an n-bit-long binary string (a1...an) into a bounded real number x ∈ [xL, xU]: where a1 is the most significant bit and an is the least significant bit [8]. This kind of transformation is just unique if the amount of levels equals (2n − 1). In other words, like its natural paradigm, the binary code is degenerated if there are more bit combinations than levels (the opposite case will not occur as the program computes the minimal amount of required bits).
Considering the above mentioned the following points are important when defining the design variables:If the experimental setup prevents an adequate choice of levels the coding can be made visible in GAME.opt. In order to assure equal processing of all variables the amount of levels among the variables should not vary too much.From the amount of levels results the length of the binary string that is needed to encode the design variables, which is shown in GAME.opt in terms of bits for each variable. From a set of M binary strings with the length l each point in the search space (X in Eq. 1) can be reached with a probability p. The relation is given in Eq. 3 and GAME.opt will display p when selecting the number of individuals (M) [9].The value of p should be higher than 0.99.As each individual corresponds to one experiment, the experimental effort will be correlated to the chosen amount of levels.The increment should be higher than the possible precision in the experiment.
Experimental results
The experimental observations y have to be specified in GAME.opt just in terms of their names and units. The following items should be considered when planning the experiments:All parameters different from the design variables have to remain fixed in all experiments throughout the optimization procedure.The experimental error should be lower than the differences that are expected to result from different levels in the design variables.
Objective functions
The objective functions in GAME.opt can be any linear combination of the experimental results and the decision variables. GAME.opt will maximize all objective values. If for example one objective is minimizing a certain experimental result, the objective function is –1 times the according result. When defining the objective functions it is important to make sure that they are independent from each other. For example it is no use maximizing simultaneously the amount of a reaction’s product and minimizing the amount of non-converted reactants at the end of a batch process, because a high product concentration is correlated to low resting concentrations of reactants. A reasonable combination is for example maximization of an experimental result while minimizing the sum of decision variables, possibly reducing the costs of a process.
Optimization procedure
Once the MOP is defined by the above mentioned steps a random generated initial population is available. This set of design variables can be exported from GAME.opt and the according experiments are performed. After the experimental results are entered in GAME.opt the next population is generated and checked for individuals that have already appeared in order to prevent repeated experiments. This iterative process is performed until satisfying results are obtained. The external population contains the efficient set recovered so far and thus constitutes the result of the optimization process. The experimental results in the external population are already assigned because their members originate from former populations.
Results
A previous version of GAME.opt was used for experimental design for media optimization (13 medium components, 20 individuals/generation, 91 bits/binary string, eight generations) and gave comparable results to another GA making use of VEGA with half the number of experiments [10]. Further process optimizations considering media composition together with process parameters are currently performed in our laboratory. In order to evaluate the software for a proper processing and its general ability to solve MOPs it was tested by means of several multi-objective test problems in such a way that the experimental results are replaced by evaluation of mathematical functions. An exemplary MOP with three objectives and six design variables is given in Eq. 4. This MOP is based on Deb et al. [11], where the development of test problems is described. The MOP in Eq. 4 was solved with GAME.opt, therefore the six design variables have been defined in the program together with the bounds, being 0 and 1 for each variable. As mentioned above the choice of the increment and thus the amount of levels is crucial for a proper processing of the variables. An increment of 0.1 will give 11 Levels, namely [0, 0.1, 0.2, ..., 1] encoded with 4 bits giving 16 levels. According to Eq. 2 an adequate increment for a given number of bits n results from:
Table 1 confronts a proper coding with 4 bits with a degenerated coding for a variable range from 0 to 1.
Table 1Four bit binary coding of the design variables in Eq. 4 for different increments (incr)Binary codeDegenerated codingDecoded value (incr = 0.1)Proper codingDecoded value (incr = 1/15)0 0 0 0000 0 0 10.11/150 0 1 00.12/150 0 1 10.23/150 1 0 00.34/150 1 0 10.35/150 1 1 00.46/150 1 1 10.57/151 0 0 00.58/151 0 0 10.69/151 0 1 00.710/151 0 1 10.711/151 1 0 00.812/151 1 0 10.913/151 1 1 00.914/151 1 1 111
Using the proper coding shown in Table 1 the GA was run eight generations with 20 individuals in the normal population and 10 individuals in the external population. Two crossover points and a mutation probability of 1% was chosen what has been identified as a reasonable default setting in other problems. As a rule of thumb the size of the external population should be half of the normal population, for which a minimal number can be estimated by Eq. 3. Table 2 summarizes the default parameter setting.
Table 2Default parameter setting for the genetic algorithmParameterValueCrossover points2Mutation rate1%Individuals in normal population (N)Eq. 3Individuals in external population (Nextern)N/2Individuals in external population after cluster analysis (Ncluster)N/2
The results are shown in Fig. 2, by means of the objective values f1, f2, f3 of the ten individuals in the external population after eight generations.
Fig. 2The objective space of the test problem lies between the surfaces. Optimal solutions are located on the unit sphere in the first quadrant. The black dots indicate the efficient set after eight generations of GAME.opt (external population). The two views of the plot point out the closeness of the efficient set to the Pareto front (a) and its equal distribution (b)
The two spherical surfaces in Fig. 2 constitute the objective space of Eq. 4.
For the minimization problem the first quadrant of the unit sphere constitutes the Pareto optimal frontier where g(x) = 0 and it is already approximated well by the ten members of an external population after eight generations (Fig. 2a). Beside the closeness to the Pareto frontier the distribution of the solutions is an important performance criteria. Figure 2b reveals the equal distribution of the ten solutions resulting amongst other things from the cluster analysis. Optimal solutions are characterized by xi = 0.5 for i = 1, 2, 3, 4, whereas x5 and x6 can take any value in the given range. Splitting the design variables in such a way allows to test the algorithm for a crucial feature of GAs, the populations diversity. On the one hand the algorithm is expected to converge fast to the optimal solutions but on the other the populations diversity must be high enough to ensure exploration of the whole Pareto optimal frontier. This fact is reflected in the box plots of the initial and the eighth generation together with their corresponding external populations in Fig. 3.
Fig. 3Box plot of the initial population (a), the first external generation (b), the eighth generation (c) and the eighth external generation (d). The boxes contain the middle of 50% of the data. The line in the box indicates the median. The bold dashed line separates variables with an optimal solution (x1, x2, x3, x4 = 0.5) from variables without optimal value (x5, x6)
Initially the individuals are randomly distributed, what is reflected by the box plots of the six design variables in Fig. 3a. A trend towards 0.5 is already visible in Fig. 3b where box plots are shown for the first external population. In Fig. 3c, d, depicting the box plots of the corresponding populations after eight generations, a clear distinction between the variables with optimal value and those without can be made. After eight generations the diversity of x5 and x6 is still high in both populations. The remaining variables show the expected divergence towards 0.5 in the external population but remain in parts distributed over a wider range in the normal population. From this result it gets obvious that GAME.opt performs well with a small amount of individuals and a short evolution on a MOP considered a benchmark of evolutionary computing.
Conclusions
In order to keep the experimental effort low, GAME.opt will use small population sizes and fewer generations compared to other GA applications, where the amount of objective function evaluations is mostly limited by the computational costs. For this reason the external population is expected to be only a rough approximation of the true Pareto optimal frontier. Nevertheless in all test problems a satisfying performance of GAME.opt was achieved in spite of the relatively low number of experiments (160 compared to 166 possible design variable combinations with the test problem presented).
In the near future, efficient parallel search strategies for experimental design are becoming more and more important in bioprocess optimization due to the availability of new parallel stirred-tank bioreactor technologies [12].

genetic algorithm
software tool
experimental design
multi-objective optimization