Empirically investigating the human role on the code smell effect

Context: Code smell is a term commonly used to describe potential problems in the design of object-oriented systems. The theory about code smells, initially presented in the nineties, is focused on characterization of different types of smells and heuristics for their detection and removal. Since then, empirical studies have evaluated the impact of the practical adoption of the code smell concept on software development. We call this issue the code smell effect. The findings of empirical studies on the code smell effect have presented inconsistencies with respect to what is expected by the theory. This phenomenon is not well understood. Objective: To enhance the understanding of the code smell effect, by characterization of problems hindering the practical adoption of the smell concept and by investigation of their origin. More specifically, we focus on the major confounding factor affecting observations on the code smell effect, which is the human role on smell detection. The human role is mostly disregarded by the theory and it strongly affects the experiments on the subject. Method: We explore the problem empirically, from two perspectives. First, we synthesize the current knowledge on the code smell effect from a set of primary studies published in the literature. To do this, we performed a systematic review based on thematic synthesis. Second, we propose the investigation of factors affecting the human evaluation of smells. To do this, we performed a family of controlled experiments. In it, we investigate an extensive number of factors affecting the human perception of smells. The factors are the use of design comprehension tool support; developer’s experience, knowledge and training; and software size. From the thematic synthesis and the family of controlled experiments, we identify challenges for the area and present them as a research agenda. Results: We find out that human evaluation of smell should not be trusted. The studies indicate that the agreement on smell detection is low. They also show that demographic attributes, such as developers’ experience, impact the agreement on smell detection. We have also concluded that, nowadays, there are no evidence supporting the use of code smells for practical evaluation of design quality. Our evidences are divergent findings correlating code smells and software attributes, such as changes on source code. In some cases, the studies converge showing that code smells do not impact some attributes. This happens, for example, with studies correlating smells and maintenance effort. Conclusion: This thesis points out to the gap between the theory and the experimental findings about the code smell effect. Our evidences indicate that the main source for this gap is the low number of studies focusing on the human role on the smell effect. The researches on the subject should focus on better understanding this issue. This does not seem the path followed by the area, which has directed its effort on researches focused on tool assessments, such as automatic detection or smell removal. In order to face the problem, we propose a research agenda, indicating that the area needs to: i) investigate the nature