The rise of open source development tools

For many years, the vast majority of tools used by professional software developers came from commercial vendors. Over the past few years, the availability of open source and non-commercial development tools has greatly increased, giving software development organizations a much broader range of alternatives for development projects. While many of these development efforts have occurred under the leadership of a collection of open source projects, there is very little coordination among the various efforts. As a result, it is now relatively easy to assemble a collection of open source development tools, but it remains difficult to create an integrated environment of such tools. This paper describes numerous open source projects and some of the possible approaches for creating an integrated environment. 1. Historical Background Open source software dates to the earliest days of programming. Numerical analysts published the source code of algorithms, making the code freely available to others. Software developed for the US Government was (and is still) available (in principle) to anyone who requests it. For all practical purposes, there was no commercial software industry before the early 1970’s, so many of today’s issues relating to software licensing and intellectual property were unexplored. (Note: the term “open source” is used here in line with the definition given by the Open Source Initiative at http://www.opensource.org/docs/definition_plain.html.) Software systems, including operating systems and programming language compilers, were typically bundled with hardware. In 1969, though, IBM decided to unbundle its software products and services from their hardware; this event can be seen as triggering the development of a market for software products. 1.1. Unix tools In the mid-1970’s, AT&T Bell Laboratories created a non-commercial distribution of the UnixTM system, distributing it to universities and other research institutions, where it quickly became the de facto standard for learning about systems software in academic institutions. The University of California, Berkeley, with funding from the US Department of Defense Advanced Research Projects Agency (ARPA), created successive non-commercial distributions of Unix, augmented by numerous locally developed tools, including the vi text editor, as well as such development tools from AT&T Bell Laboratories, including sccs and make. The source code was always part of the distributions, making it easy to port Unix and its tools to a wide variety of computer systems. Rosenberg [1] and McKusick [2] discuss this history in greater detail. Kernighan and Plauger, part of the Unix team at AT&T Bell Laboratories, were among the first to discuss tools as a separate topic [3]. They followed what has become known as the Unix “toolbox” philosophy: "Make each program do one thing well. Expect the output of every program to become the input to another, as yet unknown, program” [4, 5]. McIlroy introduced the notion of a “pipe” [6], which facilitated composition of tools. The Unix distributions included not just the underlying system, but also a broad collection of tools. All of the application programming interfaces (API’s) and data representation formats were a standard part of the documentation As the market for Unix-based commercial software tools developed in the 1980’s, some products were designed to adhere to the principles of Unix tools. Wasserman and Pircher named this philosophy “open architecture” [7,8], and it has been widely adopted among developers of tools and environments. 1.2. Open Source Platforms In addition to Unix, other open source development projects have existed for more than 20 years. Among these, the Emacs editor was developed beginning in 1978 under the aegis of the Multics project sponsored by ARPA at MIT [9]. The GNU Project, begun in 1984, initially produced a later version (GNU Emacs) and the GNU C Compiler [10], and has led the GNOME project to create a user-friendly “desktop” environment on open source software (http://www.gnome.org). The Free Software Foundation (FSF) is now the organizational sponsor of the GNU Project. FSF and UNESCO maintain a directory of free software at http://www.gnu. org/directory. The Unix system served as the intellectual foundation for several open source operating systems, including Linux ( Development tools were dominated by commercial efforts throughout the 1980’s and much of the 1990’s. Leaders among tool vendors included Microsoft, Oracle, IBM, Adobe, Borland, Macromedia, Rational Software (acquired by IBM), and Mercury Interactive. There were very few open source tools in the categories addressed by these and other commercial vendors. http://www.linux. org) (distributed under the GNU General Public License), FreeBSD (http://www.freebsd.org), OpenBSD (http://www.openbsd.org) and NetBSD (http://www.netbsd.org). Beyond the operating systems themselves, though, are numerous other open source software components that provide the framework or platform on which applications run. The Linux effort, for example, includes more than 75 related software development projects (http://www.linux.org/projects/software.html). Another widely used piece of open source software is the Apache HTTP Server, now a project of the Apache Software Foundation (http://www.apache.org). Apache, developed since 1995, has long been the most widely used web server, with many of its installations running on an open source operating system. The Apache Foundation also sponsors the Jakarta project, which creates and maintains open source solutions on the Java platform. Among the subprojects is Tomcat, the servlet container used in the official Reference Implementation for the Java Servlet and JavaServer Pages technologies, and Struts, a model-view-controller framework for Javabased web applications. Other important platform components are relational database systems, exemplified by MySQL (http://www.mysql.org), and J2EE application servers, exemplified by JBoss (http://www.jboss.org). Both MySQL and JBoss have established commercial enterprises that complement the non-commercial, freely usable versions of the software. These businesses are successfully competing against traditional closed-source commercial software vendors. The operating system, web server, DBMS, and application server, along with scripting languages (e.g., Perl, PHP, Python, Tcl, and Groovy), provide a powerful foundation for both traditional and web-based applications, making it possible for organizations to avoid the high costs of traditional commercial products for these services. By combining these software components with low-cost, commodity hardware, it becomes very inexpensive to deploy and run both traditional and webbased applications. What was missing, though, was a correspondingly attractive set of development tools for designing, building, and testing these applications. 1.3. Development Tools 2. Open Source Tools In the past five years, though, the number of projects to create open source development tools has grown tremendously, driven by the overall success of open source platform and enabled by projects that allow developers from around the world to collaborate and contribute to the development of a tool. The economic downturn in developed countries, with the end of the “dot-com” bubble, led to a significant decrease in purchases of licenses for traditional commercial software tools. Even some commercial vendors have now released source code for some or all of their products. For example, Microsoft has established a Shared Source Initiative (http://www.microsoft.com/resources/sharedsource/defaul t.mspx) and has released their Windows Installer XML under this Initiative. 2.1 Open Source Project Sites Interest in open source development has led to the creation of numerous community sites that support collaborative software development projects, many of which involve development tools. These projects are typically characterized by a collaborative, consensus based development process, with a designated project leader and a small team of individuals who are authorized to “commit” their code to the evolving project. Others can review the code and can try out the software, making suggestions for changes. Among the open source development sites are: 1) Apache – produces the Apache HTTP server and the various Jakarta Project solutions for the Java platform (http://www.apache.org) 2) Mozilla.org – manages the Mozilla, Firefox, and Camino web browsers, the Bugzilla issue tracking system, and the Thunderbird email client (http://www.mozilla.org) 3) SourceForge – the world’s largest site for open source development, currently hosting more than 80,000 separate projects, including some involving tools (http://sourceforge.net) 4) CVS Home – oversees development of the CVS version control system (http://www.cvshome.org) 5) Eclipse – a universal, open, extensible tools platform created with support from more than 50 member companies; current projects include a C/C++ IDE, a modeling framework, and a web tools platform (http://www.eclipse.org) 6) Mono – a Novell-sponsored project to create an open-source version of Microsoft . Net (http://www.go-mono.com) 7) WebDAV – web-based distributed authoring and versioning (http://www.webdav.org) 8) Tigris – open source tools for project management and UML modeling (http://www.tigris.org) 9) JUnit – Java automated testing tool hosted on SourceForge (http://www.junit.org) 10) NetBeans – the project that develops the NetBeans IDE (http://www.netbeans.org) Several of these projects use CollabNet SourceCast (http://www.collab.net) or the VA Software SourceForge environment as the underlying enabling technology. These collaboration mechanisms provide a project repository along with version control, configuration management, mail, and project management support. These projects largely exist independently of one another, even within a specific site.