Trace alignment , a procedure for finding common activities and deviations in process executions, does not have a well-established framework for evaluation. A common alignment evaluation tool-reference - based methods - is not applicable if a reference alignment, or ground truth, is not available. On the other hand, reference -free evaluation methods currently are not able to adequately and comprehensively assess  alignment quality. We analyze and compare the existing evaluation methods, identify their limitations, and propose improvements . We introduce modifications to two reference-free evaluation methods , improving their ability to assess alignment quality. We summarize the parameter selection for these modified methods and analyze their results. We also tested these evaluation methods on the alignment of a trauma resuscitation process log .
Trace alignment algorithms have been used in process mining for discovering the consensus treatment procedures and process deviations. Different alignment algorithms, however, may produce very different results. No widely-adopted method exists for evaluating the results of trace alignment. Existing reference-free evaluation methods cannot adequately and comprehensively assess the alignment quality. We analyzed and compared the existing evaluation methods, identifying their limitations, and introduced improvements in two reference-free evaluation methods . Our approach assesses the alignment result globally instead of locally, and therefore helps the algorithm to optimize overall alignment quality. We also introduced a novel metric to measure the alignment complexity, which can be used as a constraint on alignment algorithm optimization. We tested our evaluation methods on a trauma resuscitation dataset and provided the medical explanation of the activities and patterns identified as deviations using our proposed evaluation methods .
R
, a procedure for finding common activities and deviations in process executions, does not have a well-established framework for evaluation. A common alignment evaluation tool-reference - based methods - is not applicable if a reference alignment, or ground truth, is not available. On the other hand, reference -free evaluation methods currently are not able to
algorithms have been used in process mining for discovering the consensus treatment procedures and process deviations. Different alignment algorithms, however, may produce very different results. No widely-adopted method exists for evaluating the results of trace alignment. Existing reference-free evaluation methods cannot
A
the
R
analyze and compare
analyzed and compared
R
identify
identifying
R
propose improvements . We introduce modifications to
introduced improvements in
R
, improving their ability to assess
. Our approach assesses the alignment result globally instead of locally, and therefore helps the algorithm to optimize overall
R
summarize the parameter selection for these modified methods and analyze their results. We also tested these
also introduced a novel metric to measure the alignment complexity, which can be used as a constraint on alignment algorithm optimization. We tested our
D
the alignment of
R
process log
dataset and provided the medical explanation of the activities and patterns identified as deviations using our proposed evaluation methods