## 5 Human-Assisted Within-Show Diarization
### Within-Show Question Generation Module  
When processing a single file, we propose to generate questions by following the architecture of the dendrogram produced by HAC (Figure 2a). HAC is done with no prior on the number of clusters so the threshold is empirically determined on the development set. This threshold separates the dendrogram in two parts (above and below the threshold). From this point, the same question can be asked to the human expert for each node of the dendrogram: \"_Do the two branches of the node belong to the same speaker?_\". A \"yes\" answer from the human expert requires either to merge the two branches of a node above the threshold (merging operation) or, if the node is before the threshold, to leave the branches as they are (no splitting is required here). In case of a \"no\" answer, a node above the threshold would not be modified (no merging required) and the two branches of a node below the threshold would be separated (split operation). One must now determine which node to ask about and when to stop asking. To do so, we rely on the distance between the threshold and the nodes, referred to as _delta_ to differentiate with distance between \(x\)-vectors. Examples of those _delta_ are labeled \(d1\) to \(d6\) on Figure 2a. Nodes are ranked in increasing order according to their absolute _delta_ value. We propose to ask questions about the nodes in this order, and consider two different stopping criteria. First, a **two confirmation** (\(2c\)) criterion illustrated in Figure 2b, in which we assume that if a node above the threshold is confirmed by the human expert to be separated then other nodes above it, with higher _delta_ will not be investigated. Similarly, if one node below the threshold is confirmed by the human expert to be merged, the other nodes, lower in the dendrogram, will not be investigated. Second, we consider a per-branch strategy (_All_) to explore the tree in more details (see Figure 2c). Initially, a list of all nodes, according to their ranked _delta_, is prepared for investigation. Applying the _All_ criterion, each time a question gets a confirmation (i.e., the human validates the system decision and no correction is required), the list of ranked nodes to investigate is updated by removing some of the following nodes as follows: (i) for a node lower than the threshold, if the human expert confirms the merge, its children nodes in the two branches will be removed from the list of nodes to investigate. (ii) for a node above the threshold, if the human expert confirms the split, its parents nodes will be removed from the list of nodes to investigate. For example in Figure 2c, \(d4\) will not be investigated because \(d3\) has been confirmed as a split. The \(2c\) criterion relies on a high confidence in the _delta_ ranking (the estimation of the distance between \(x\)-vectors) and strongly limits the number of questions, while the _All_ criterion leads to more questions and thus a finer correction of the dendrogram. Figure 2: The change of HAC dendrogram after interaction with human expert. In (**a**), the nodes located below the threshold (\(d1\), \(d2\), \(d5\) and \(d6\), ranked by increasing distance to this threshold) will be investigated for splitting correction, and the nodes located above the threshold (\(d3\) and \(d4\), ranked by increasing distance to this threshold) will be investigated for merging correction. The \(d1\) node in (**b**) (and \(d1\) and \(d5\) nodes in (**c**) has been modified according to the human correction. To facilitate the work of the user answering the question, we consider that the HASD system proposes two audio segments (_samples_), for the user to listen to; one for each branch of the current node. Each branch can link several segments, even for nodes located at the very bottom of the tree (remember that, due to the sequential HAC clustering process, leaves of the dendrogram are clusters linked by the BIC-HAC clustering). The system must select the two most representative or informative _samples_.