Squinting at the data : Investigating software artifact provenance using KISS techniques

Since the appearance of Albrechts pioneering work, function points have attracted significant attention from the industry. In their work, project managers can benchmark function point counts obtained for their projects against large publicly available datasets such as the ISBSG development & enhancement repository release 11, containing function point counts for more than 5000 projects. Unfortunately, larger amount of functionality as reflected in the function points count does not necessarily correspond to a more significant development effort. In this paper we focus on a collection of ISBSG projects with a similar amount of functionality and study the impact of different project attributes on the development effort. In our study we consider the ISBSG development & enhancement repository release 11, the largest publicly available dataset with function point counts, containing data about more than 5000 projects developed in a variety of different countries using a variety of different design and development techniques [1]. The ISBSG repository contains information about 118 different project attributes, including its functional size as well as organizational (e.g., scheduling, development team size and its productivity), technical (e.g., architecture and the main programming language), and problem-specific attributes (e.g., business or application area). Data is provided by the project owners themselves. Functional size is for most of the projects measured by applying such methods as IFPUG function points (3799 out of 5052 projects or 75.2%) [2], hence we solely focus on the IFPUG projects. For the IFPUG projects, the ISBSG repository contains data on the development effort. First, we exclude projects that report the effort figures only for some project phases. Second, while some projects report on the recorded effort, some other projects report on the estimated effort based, e.g., on the amount of functionality. To ensure validity of our study we solely consider projects that report on the actual recorded effort. Furthermore, we focus only projects that record the effort in staff hours (as opposed, e.g., “productive time”) and that only record time spent on software development, including project management and project administration, but excluding non-project specific supporting activities such as control and audit or hardware support. Overall, excluding projects that report the effort figures only for some project phases, that report estimated rather than recorded effort, that report recorded effort expressed in some other unit than staff hours, or that include the effort dedicated to non-project specific supporting activities, reduces the number of considered projects to 1661. Finally, as the data in the ISBSG repository is provided by the project owners themselves, it might become polluted by imprecise or unreliable values. Therefore, ISBSG quality reviewers assess the soundness of the data submitted. The result of the assessment ranges from “A” to “D”, where “A” indicates that the data “was assessed as being sound with nothing being identified that might affect its integrity”, while for “B” the data “appears sound but some aspects might have affected the data or count integrity”. Assessment “C” indicates impossibility of assessment due to incompleteness of the data provided, while little credibility should be given to data assessed “D”. Restricting our attention to the “A”-projects reduces the number of the eligible projects to 84, while 1609 projects remain if both “A”and “B”-projects are considered. Hereafter we consider both “A”and “B”-projects. As mentioned, for 2 4 6 8 10 2 4 6 8 1 0 1 2 log(Adjusted Function Points) lo g (S u m m a ry W o rk E ff o rt ) each project the ISBSG repository contains information about 118 different project attributes including the summary work effort, functional size (measured using the so called unadjusted and adjusted function points counts), as well as organizational, technical and problemspecific attributes. We prefer adjusted function points count to unadjusted function points count (cf. [3]), and consider the following project attributes: primary programming language, language type, organization type, intended market, year of project, development type, platform, and architecture. Plotting the summary work effort against the adjusted function points count reveals presence of four outliers, projects containing more than 5000 function points. Moreover smaller projects up to 500 function points cover the entire range of work effort. The log-scale plot recommended in [4] shows a