# Revisiting Problem-Solution Co-Evolution in the Context of Team Conceptual Design Activity
## 2 Materials and Methods
### Measures and Visualisations
Namely, the majority of fine-grain studies of design activity utilise protocol analysis to decompose the process into small chunks (process elements) [35], and the resulting protocols (instances of process elements) are usually analysed in terms of their duration, frequency, proportions, sequences, and probabilities of moving from one process element to another (for the most relevant examples, please consult [24; 41; 42; 46; 50; 66; 67; 70; 71; 72; 73; 74]). Proportions and probabilities are particularly useful as they enable straightforward inter-study comparisons. Hence, a similar approach was adopted here, and three dependent sets of variables were defined as follows:  
* Proportions of state transitions: Instances of ASE transitions within the problem and the solution space are counted and normalised (divided by their total number) in order to calculate the proportion of each type of transition in the time span of the team design activity (or fragment of the activity). Proportions of state transitions (measured in percentages) provide insight into the general state-transition nature of the investigated activity, in terms of the team's orientation towards analysing, synthesising, or evaluating problem and solution entities. When analysing team design activity, it is necessary to capture the appearance of six basic transitions. These six types can, of course, be aggregated into ASE or problem- and solution-related design operations. * Proportions of state-transition sequences: Instances of two or more (depending on the specific aspect of analysis) consecutive state transitions are counted, and the overall distribution is normalised to calculate the proportions of different combinations of sequences of two or more state transitions. Proportions of state-transition sequences, which are also measured in percentages, enable the identification of most common state-transition patterns exhibited when designing in teams. The total number of possible sequence combinations between the six basic transitions (ASE in problem and solution space) is 36. These 36 sequence combinations between two transitions can be aggregated into nine combinations of moves between analysis, synthesis, and evaluation and four combinations of moves in between the problem- and the solution-related transitions, thus providing higher-level process measures. * Probabilities of state-transition sequences: The counted instances of two consecutive state transitions can simply be transformed into probabilities of one particular state transition following another. These probabilities, again measured in percentages, are useful for both comparing and simulating experimental datasets, particularly when interpreted as probability matrices in Markov processes [75]. Variables related to the proportions of state transitions can be utilised for describing the general nature of team design activity or its fragments, whereas the variables related to proportions and probabilities of state-transition sequences are intended for modelling the patterns of state transitions during team designing. The measurements associated with the proposed variables can be sampled from the team activity protocols that must previously be captured using the state-transition model. Simply put, the proportion measures sampled from the experimental datasets enable quantification of the design process and statistical testing of hypotheses. The reliability of the used measures [69] must be ensured as part of the data collection process. In the case of conducting a protocol analysis study, the level of reliability must be determined by calculating the inter-rater (inter-coder) reliability [73; 76]. The following notation is used for the proportion variables: \(p_{\text{A}}\) for the proportion of analysis, \(p_{\text{S}}\) for the proportion of synthesis, and \(p_{\text{E}}\) for the proportion of evaluation.