Factors leading to the success and sustainability of software process improvement efforts

Many software development organizations today are keen on improving their software development processes in order to develop software products faster, cheaper or better. For that reason, Software Process Improvement (SPI) has received significant attention from the research community over the last few decades. Process maturity models have become widely known for benchmarking software processes against predefined practices and for identifying processes to be improved or implemented, whereas process improvement approaches were developed for guiding the actual process improvement process. However, despite a wide number of provided guidelines on how to standardize the processes and how to run process improvement efforts, only a few SPI initiatives have succeeded. About 70% of the SPI initiatives fail and a significant number do not even get started. Many studies argue that the success of the SPI initiatives is dependent on the organizational, social and managerial aspects of process improvement. Those aspects however are not sufficiently covered by the existing SPI approaches and models. The little knowledge on organizational, social and managerial aspects of SPI that is available is mostly scattered across the domain. Hence, there is lack of a holistic overview of the current SPI domain that provides sufficient coverage of organizational, social and managerial aspects of SPI.This thesis has explored the organizational, social and managerial aspects of SPI and placed them into the context of the SPI domain. Its main research result is Software Process Improvement Framework (SPIF). The framework provides an overview of the SPI domain and positions theories representing organizational, social and managerial aspects of SPI in the context of existing SPI approaches, models, methods and practices. SPIF is based on the existing theoretical framework for SPI environment proposed by Sami Zahran. The SPIF framework has been additionally complimented with four additional outcomes of this study. Those are: 1) a list of organizational, social and managerial factors facilitating SPI effort, 2) a list of contextual factors impacting process change, 3) a process model for guiding software method adoption, and 4) a checklist representing the properties of successful and sustainable SPI projects.The research was based on a strong industrial cooperation. As many as thirty software development organizations were involved in this research. Methodologically, the research was conducted in line with the inductive reasoning, which guided the research into building the knowledge from empirical studies. However, at some stages of this research, literature studies were incorporated. The main research methods of this study are action research and case studies, whereas data collection methods are primarily structured interviews, participatory observations and surveys.The thesis concludes that implementing a recommended software development processes or practices using well defined SPI approaches is not enough. In order to implement successful and lasting process improvement, organizations also need to consider organizational, social and managerial aspects of SPI. The SPIF framework and other results of this thesis may significantly benefit software development organizations that plan to conduct software process change, or have already done it. These organizations may use SPIF for getting an overview of the process improvement process and the theories, methods and tools that should support it. The other results of this thesis can be used for: 1) incorporating organizational, social and managerial aspects in process changes, 2) for adapting process improvements in various organizational contexts, 3) for guiding adoptions of new software development methods, and finally 4) for evaluating and improving process improvement efforts.