Evaluating and Improving Semistructured Merge
Channel: Splash Conference 2017
Date: 2017-10-30

Guilherme Cavalcanti, Paulo Borba, Paola Accioly
While unstructured merge tools rely only on textual analysis to detect and resolve conflicts, semistructured merge tools go further by partially exploiting the syntactic structure and semantics of the involved artifacts. Previous studies compare these merge approaches with respect to the number of reported conflicts, showing, for most projects and merge situations, reduction in favor of semistructured merge. However, these studies do not investigate whether this reduction actually leads to integration effort reduction (productivity) without negative impact on the correctness of the merging process (quality). To analyze this, and better understand how merge tools could be improved, in this paper we reproduce more than 30,000 merges from 50 open source projects, identifying spurious conflicts reported by one approach but not by the other (false positives), and interference reported as conflict by one approach but missed by the other (false negatives). Our results and complementary analysis indicate that, in our sample, the number of false positives is significantly reduced when using semistructured merge, and we find evidence that its added false positives are easier to analyze and resolve than those reported by unstructured merge. However, our evidence shows that semistructured merge might lead to more false negatives, and we argue that they are harder to detect and resolve than unstructured merge false negatives. Driven by these findings, we propose an improved semistructured merge tool that further combines both approaches to reduce the false positives and false negatives of semistructured merge. We find evidence that the improved tool, when compared to unstructured merge, reduces the number of reported conflicts by half, has no extra false positives, has at least 8% less false negatives, and is not prohibitively slower.