Mining Software Repositories to Guide Software Development

Software repositories (such as source control repositories) contain a wealth of valuable information regarding the evolutionary history of a software project. In this research we recover such historical data and present several techniques and approaches to guide managers and developers working on large software systems. We validate our work empirically using data based on over 60 years of development history for several open source projects. 1 PROBLEM STATEMENT As the size and complexity of software systems increases, software practitioners find themselves facing many challenges to deliver high quality software on time and within budget. Software managers are continuously estimating the quality of their software and predicting its reliability to ensure that no faulty software is released. Moreover, they endeavor with varying degrees of success to wisely allocate their limited testing and development resources to the parts of the most appropriate parts of the code. Unfortunately, in many cases such attempts are based on ad-hoc techniques and rough approximations. Their success depends on their intuition, experience and chance. Similarly, software developers spend a large amount of their time trying to comprehend source code as they modify it to implement the evolving requirements of the customers. They perform such complex tasks using simple text editors and tools such as grep. Such primitive tools are limited by the size of the software system and the amount of information a person can keep track of while jumping around the source tree [16]. The evolutionary history of a software project offers a great opportunity to assist these practitioners. In this work, we propose a set of techniques and tools to assist software practitioners by mining historical data about the code development process. 2 RESEARCH HYPOTHESIS The code development process represents the patterns of modifications to the source code of a project. These modifications are done by the developers to implement new features and repair faults. Whereas the current code base represents a static view of the current state of the system, the code development process provides a rich and evolving historical record of the code. It plays a central role in the production of a software system where several facets interact heavily and affect each other (see Figure 1). Changes, complexity, lack of understanding, and concerns in one facet flow to other facets and affects them. For example, schedule pressure, requirements, and team structure affect the development process. As the complexity of the development process increases, it negatively affects the source code whose complexity increases as well. Over time the complexity of the source code and its design will increase, in return the complexity of the development process increases creating a feedback loop. Development Process Code/Design Requirements Team (size/structure) Problem Domain Market/ Schedule Pressure Figure 1: Interactions Among Facets of a Software Project By studying patterns of modifications to the source code, we hope to achieve a better understanding of the software system and the development process in general. We claim that: Mining the evolutionary history of a software system can assist us in guiding software practitioners (managers, developers, etc.) to build better soft-