# Revisiting Problem-Solution Co-Evolution in the Context of Team Conceptual Design Activity
## 3 Results
### The Analysis-synthesis-evaluation Sequence  
The second set of hypotheses concerns the dominant sequences of ASE state transitions in design space exploration. The sequences were investigated by determining the probabilities of one state transition following another; that is, what is the probability of either analysis, synthesis, or evaluation to follow the previous type of state transition (Markov analysis [75]). First, the different combinations of adjacent pairs of state transitions in the protocol fragments of all teams were counted (e.g., A\(\rightarrow\)A: analysis followed by analysis, A\(\rightarrow\)S: analysis followed by synthesis, A\(\rightarrow\)E: analysis followed by evaluation, etc.). These counts were normalised into percentages by dividing the count of a particular sequence with the total number of sequences that share the same starting state transition. The normalised results (see Table A2 in Appendix A) represent the probabilities of performing analysis, synthesis, and evaluation, given the previous state transition. The probabilities across all activity fragments and all teams were summarised using boxplots in Figure 6.  
The boxplots show that the most probable state transition following analysis is synthesis and that the most probable state transition following synthesis is analysis. It is, however, not clear whether evaluation is most probable to be followed by analysis or synthesis. Additional statistical tests were performed to investigate the significance of difference between these probabilities. In their investigation of the analysis-synthesis-evaluation sequence, [PERSON] et al. [39] utilised the t-test to compare the pairs of sequence probabilities. The here-presented statistical analysis differs in two ways. First, it also takes into account the probabilities of performing a sequence of two same state-transitions (analysis-analysis, synthesis-synthesis, and evaluation-evaluation). Second, it utilises a non-parametric statistical test, due to the violated assumption of normally distributed data for some of the sequence proportions and probabilities. Namely, while the Shapiro-Wilk test did not reject the assumption of normally distributed data regarding the ASE sequences, the [PERSON]'s test rejected the assumption of homogeneity of variance for all three analysis cases shown in Figure 5. For that reason, the Friedman non-parametric test was used to test whether there exists a significant difference between the probabilities sampled across the team design activities, and the Wilcoxon signed-rank test was used in the post hoc analysis in order to determine whether a significant difference exists between the specific pairs of sequence probabilities. The Friedman test indicates significant differences between probabilities within all three groups: state transitions following analysis (\(\chi^{2}\) = 20.891, \(p\) < 0.001), state transitions following synthesis (\(\chi^{2}\) = 24.333, \(p\) < 0.001), and state transitions following evaluation (\(\chi^{2}\) = 32.109, \(p\) < 0.001). The Wilcoxon signed-rank test (paired per the same starting state transition in a sequence) was then conducted to identify which pairs of sequence probabilities differ significantly within the three groups. In order to reduce the likelihood of Type I error, the Bonferroni correction was applied to the results of the Wilcoxon test. Significant differences at the \(\alpha\) = 0.05 level were found between the following probability  
Figure 6: Boxplots of sequence probabilities based on the previous state transition. pairs: A\(\rightarrow\)A and A\(\rightarrow\)S (\(p\) = 0.002), A\(\rightarrow\)S and A\(\rightarrow\)E (\(p\) \(<\) 0.001), S\(\rightarrow\)A and S\(\rightarrow\)S (\(p\) = 0.006), S\(\rightarrow\)A and S\(\rightarrow\)E (\(p\) \(<\) 0.001), E\(\rightarrow\)A and E\(\rightarrow\)E (\(p\) \(<\) 0.001), and E\(\rightarrow\)S and E\(\rightarrow\)E (\(p\) \(<\) 0.001).