Source: https://livablesoftware.com/large-community-software-development-erc-proposal/
Timestamp: 2019-04-22 12:24:30+00:00

Document:
Following up on our introductory post in the topic I’m now “releasing” the full B1 Research Proposal document I submitted to the ERC Consolidator Grant 2016 focused on the study of open source communities.
We live in a software-enabled world. Software is everywhere, in your laptop, your phone, you car and even (sooner than later) your toaster. Global cost of software development is estimated to be over one trillion dollars making it a crucial market for Europe’s ICT initiatives.
Much of this software is critical for the daily activities of our society and has a large community behind it, comprising thousands of contributors but also millions of users that must be listened to as well. This should be especially true for software built following the principles of Open Source Software (OSS) typically developed in a collaborative manner via online code hosting platforms like GitHub.
In theory, OSS is of better quality thanks to this higher community involvement (at different levels: submitting bug reports, feature requests, giving feedback, contributing code…). Luckily, most of the crucial software for our society is OSS (like Apache Server, Firefox, Linux or WordPress). In practice, though, many OSS projects suffer from a lack of transparency and democracy, fail to attract and manage contributors and, in general, are unable to properly respond to their users’ needs. This hampers their future success and will impact the growth of Europe’s ICT.
The goal of this project is to transform software development into a real community-driven process by providing an online collaborative platform where a software community at large (i.e. including its users) can effectively participate and be managed in order to make joint decisions in the open to ensure the long-term sustainability of the project. This will require solving a number of research challenges around the human and social aspects of software development. Therefore, the project will build a unified interdisciplinary framework combining techniques from software mining and analytics with methods borrowed from political science, sociology, and economics.
We live in a software-enabled world and open source software is a key player in it: “Software is everywhere today, yet its instrumental role in the modern digital economy is often overlooked. With market revenues of over €200 billion in Europe software is the largest and the fastest growing segment of the ICT market … Open source software (OSS) is now playing a significant role in this Software economy. A number of OSS specific actions could contribute to growth in Europe, jobs creation and improvement of the European Software imbalance ” – European Software Strategy Report.
These numbers and vision clearly convey the importance of software development and, in particular, OSS development in the European economy (and, in fact, our daily life, each of us interacts with OSS every single day even if inadvertently). According to the Open Source initiative: “OSS development is a development method that harnesses the power of distributed peer review and transparency….The promise of OSS is better quality, higher reliability, more flexibility, lower cost, and an end to predatory vendor lock-in“. This level of quality is due to the active participation of the community. This is also the key proposal of the well-known essay “The Cathedral and the Bazaar”  where the author contrasts two development models: the Cathedral model where code is developed by a restricted set of developers and the Bazaar model where development is a collaborative endeavor and users are co-developers constituting altogether a very large global community of people with different profiles. Indeed, this “co-developer” role doesn’t mean users contribute code, it highlights the fact that users are key members of the software community, have a say in it and can contribute in any form or shape they can, e.g. submitting bug reports, feature requests or just giving feedback on any aspect of the software. This is different from end-user development approaches  that pretended to convert users in semi-developers to adapt themselves the software alone.
Unfortunately, this does not reflect the reality of OSS development and therefore the potential benefits of OSS to the European society may never happen. Reality shows that many OSS projects are closer to the Cathedral model than the Bazaar one. I manually analyzed the twenty-five most popular projects in GitHub and found out that only one (4%) explicitly described how user contributions would be managed (with another 28% giving partial hints). This means that 68% had no explicit governance model. Absolutely none of them were democratic (i.e. end users could not vote in any way not even to elect people to represent them). In fact, the only one describing its decision-making process stated that “this project follows the timeless, highly efficient and totally unfair system known as Benevolent dictator for life”. Clearly, not what is common practice in the rest of community aspects of our society. And this is not the only problem. Most projects struggle to attract contributors and to properly manage their massive communities of developers and users. In fact, we can conclude the OSS model is broken with many projects failing and getting abandoned in the very early stages (see  for some statistics). Therefore, alternative software production models deserve to be explored now.
I argue in this proposal that to improve software quality (in the broadest sense of the word, i.e. including product-market fit) we need to shift the focus of our software engineering research from a code-centric focus to a people-centric one. This shift will be achieved by implementing an ambitious multi-dimensional and cross-disciplinary research agenda that will bring to the software field expertise available in other academic disciplines. This is obviously a challenging task since it will involve transforming the way software is developed, making the process more open (now for real!) and community-driven. Still, software has largely contributed to make our world more social (e.g. enabling the social networks or the sharing economy services) and democratic (e.g. e-democracy and voting systems). I believe it is time we explore how these aspects can benefit software development itself.
The software research community has been chasing forever the silver bullet that will fix all problems in software engineering . Recently, the availability of a massive dataset of software project data in repositories like GitHub (with over 30 million projects, even if data needs to be taken with a grain of salt – ) has opened new research opportunities focusing on mining such repositories for valuable insights on good software development practices, specially wrt open source projects. We have performed a systematic literature review of these papers resulting in the selection of over 100 papers that have been analyzed and classified to detect the open research challenges in the software domain. Herein, we present a summary of this work, validating the need for this research proposal.
Published papers analyze software projects from different angles but mostly with a code-centric view, meaning that they focus its analysis on the projects’ source code by analyzing, for instance, (1) the use of programming languages (e.g., , ), (2) the type of license they apply (e.g., , ), (3) the folder structure of the project  or the potential vulnerabilities and complexity of the code (e.g., , ). Others focus on more methodological aspects covering testing practices (e.g., , ), refactoring (e.g., ) or pull requests (e.g., , ). This is also true for several European funded projects on OSS-related areas like MANCOOSI , OSSMETER or MARKOS.
Only a few works analyze the social part of the software development process, trying to understand how developers are internally organized and work together in the project. There are studies on the team diversity (e.g., , ) and composition (e.g., , ). Community dynamics are analyzed looking at the interactions between community members and the project or among members themselves. The former category includes works that analyze the first impression formation (e.g., ), using projects for hiring new people (e.g., ), onboarding (e.g., ) and social coding (e.g., use of the social services of GitHub to track activity in projects of interest ). The latter includes works studying the social and technical factors that motivate people to contribute to a given project (e.g., ), algorithms that recommend developers to open tasks (e.g., ) and their role in promoting together the project itself (e.g., ).
It has strong difficulties in attracting contributors with most projects having only one or two contributors.
In this proposal we aim at developing original research contributions for each one of these challenges.
with the benefits of a faster and higher-quality software production and, importantly, a better alignment with the needs of the community at large. The following figure tries to illustrate this change of perspective, highlighting how we go from the current developer centric view (kind of a meritocracy where only core developers have the right to decide) to a community that now collaborates together and has the tools it needs to manage this collaboration in an optimal way.
This community-driven process will be enabled by borrowing and adapting to the software development field techniques from the domains of political science, sociology (e.g. social/behavioural informatics), economics and ecology that had been studying a diverse range of communities for centuries, and combining them with core software techniques for mining of software repositories, constraint solving  and language design, among several others.
Employ software mining techniques to conduct a systematic study of current governance models in OSS projects. Complement it with interviews to project members to better understand the reasons behind those choices.
Develop a domain-specific language to enable OSS projects precisely define their governance model extending the basic strategies covered in . Given their explicit definition, rules could even be automatically enforced and its execution registered for future traceability (e.g. who voted for this at that moment in time?).
Adapt different democracy models (representative, direct, liquid, …) and other political systems to the specific context of OSS to empirically test the best model for OSS projects, depending on the project characteristics.
Assist projects transition to more democratic practices, if so desired by them. This may involve for instance the automatic suggestion of possible internal leaders (based on their repository activity) to represent groups of users in elections for intermediate technical committees in a representative democracy scenario. Aspects like the Gini index  for equality distribution and the quality of the online deliberation, inspired from will also play a role.
Develop goal models  for each participant profile in OSS to better understand their motivations.
Propose innovative contribution models. We believe OSS can be regarded as an example of a matching market (markets where money is not the main factor ) and therefore we can adapt retribution strategies successful in other matching markets to the OSS one. Examples would be to replicate the idea of time banks or donor chains (I help you if you help somebody that can help me).
Apply gamification principles to OSS to increase the level of contribution of current members.
Identify potential new contributors that have the skills a OSS project is looking for by analyzing and cross-profiling people’s public profiles and behavior in social networks reusing expert finding techniques like   , . This may also be used to reduce the gender gap  and increase team diversity.
Visualize the community network as a typed directed multigraph (where edges would denote several kinds of interactions between the members) and adapt well-known graph-based algorithms to identify subcommunities, leaders, low density areas and so on. Then project owners can react to solve this, e.g. by “building bridges” between the subcommunities or inviting people to specially scarce areas in the project.
Define acceptable thresholds and ranges for some social metrics in OSS (e.g. bus factor  or the ratio between external and internal contributors) depending on the project size and domain to evaluate the “health” of the community. The ranges would come from the analysis of a representative set of “successful” projects and typical values in other fields like human ecology.
Adapt review aggregator and sentiment analysis techniques to summarize long conversational exchanges to facilitate in order to let everybody easily follow relevant project discussions.
Projects do not grow in isolation. All the dimensions described above need to be extended to deal with cross-project interactions since project dependencies take place not only at the technical level but at the human level : projects compete for the same resources (e.g. developers’ time) and have cascade effects on each other. I will model this as a constraint optimization problem  aimed at finding an optimal assignment of resources to projects.
All techniques described above will be implemented and released as part of an online collaborative platform. Once built, this platform will enable a software community at large to effectively participate in the development process according to the practices and principles developed in the project. The platform will be built by ourselves as part of the project but following the “eat your own dog food” principle, it will also be released as an open source project in itself and therefore open to contributions and suggestions from the open source community. To avoid reinventing the wheel, the platform will be built on top of GitHub (or another similar hosting platform) and provide connectors with external add-ons (e.g. forums, mailing lists, external bug trackers) to be used as additional information sources for the analysis tasks of the project.
Open source is reaching its tipping point where, more than ever, even the most powerful tech companies and entrepreneurs are embracing open source  while the number of projects grows exponentially (GitHub went from 10M projects to over 30 in two years) alongside their impact on the global economy and society. And the OSS community itself is quickly realizing that at this scale, better collaboration is a must (e.g. see this open letter  to GitHub promoted by a group of maintainers of OSS projects frustrated with the limited collaboration capabilities of the platform).
My broad range of research interests and background (in software modeling , including goal modeling , formal methods , software analysis and mining , domain specific languages  and different kinds of empirical studies e.g. , to give a few examples ) covering the skill set required by the project.
My preliminary work on some of the research topics, e.g. the first version of a specific language for governance of OSS projects  or our study of the problems in attracting contributors , plus expertise on conducting research on software mining and the GitHub platform (e.g. , ).
My long term interest in several open source communities. Beyond GitHub, we are deeply involved in the Eclipse open source community (see ) and I am personally involved in the WordPress ecosystem .
Scientific impact: Transforming software development. The techniques developed in the project will have a substantial impact in the way that software projects are developed, analyzed and evaluated and will shed some light on the reasons why some projects are successful while others are not. I am confident that this project can open a new area of research where more and more knowledge from other completely different fields is deemed useful in Software Engineering and brought to it, something that so far has been done only occasionally.
Impact in proprietary software development. Private companies can benefit from many of the techniques developed as part of this, e.g. to evaluate the performance of their employees or get feedback from users. In fact, it has been shown that adopting OSS practices, a process called inner source, is beneficial for companies .
Outside the software world: impact on organizations. The work on formalization and monitoring of governance models (goal 1) is of interest for any kind of organization that wants to be transparent. Moreover, many of the social analysis techniques (goal 3) could be easily redefined to be applied on other communication platforms (e.g. forums, email threads) and not just on software-specific repositories. For instance, modeling the governance of NPO/NGO organizations could help us evaluate and compare their openness. Same for political parties and even countries.
Helping other research projects. A key long-term impact of the project should be its contribution to accelerate the advance of research in the field. Therefore, as part of the project, I will have as explicit goal the development of a series of artefacts useful to other research teams. For example, we will develop a representative sample builder  of projects in GitHub to be used as a benchmark when comparing results of different research works.
CODE will adhere to the Design-Science Research (DSR) paradigm . DSR is a problem-solving paradigm for activities dealing with the construction and evaluation of technology artifacts as well as the development of their associated research theories. Besides, CODE will make extensive use of empirical research methods both quantitative (e.g. in the automatic mining of repositories) and qualitative (e.g. semi-structured interviews to gather the motivation and requirements of participants in OSS projects and validate the results). The project will be conducted in an incremental and iterative manner  where at each iteration new advances in each of the project goals will be achieved. Validation of project advancement will be performed at the end of each iteration via the practitioners board (see “Resources” section) and via the automatic measurement of pre and post values of a number of metrics for a set of benchmark projects (both existing and created from scratch to be used as guinea pigs) monitored during the full duration of CODE.
Sketch of the work plan.
Broad range of research techniques required to accomplish the project goals (Probability: Low / Impact: Low). I have some previous experience with all the required techniques. Other members of the team will contribute also their strong technical skills in some of these areas minimizing this risk.
Cross-disciplinary nature of the project (Probability: Low / Impact: Medium). My institution’s name is “Internet Interdisciplinary Institute”, meaning that it has interdisciplinarity at its heart and favours as much as possible cross-domain scientific exchanges. A project like this is, then, a perfect fit for the institution and its strengths, and will have its complete endorsement and network of researchers to complement our skills and knowledge.
Dependency from open source repositories to get the data needed for the analysis (Probability: Low / Impact: Low). The project has a technical dependence to GitHub as the dominant code hosting platform nowadays. However, if GitHub decides to close down or change its business model, others (Bitbucket, Google code,…) will immediately take the opportunity to fill this market and we could easily adapt to their platforms to continue the project.
Little engagement of the OSS community, especially to test and validate the results of our research (Probability: Low / Impact: Medium). I have been able to recruit industrial participants in the past using my blog as a medium. We can also ensure the involvement of our many contacts in the GitHub, WordPress and Eclipse communities. Besides, we are already discussing (e.g.) these research ideas in the open to gauge the interest of the community (also clearly expressed in this kind of initiatives, e.g. ) and learn their main concerns.
I, as PI, will dedicate 70% of my time to CODE during the whole length of the project and will benefit from the support of my research team (ten members right now). Additionally, and given the cross-disciplinary nature of the project, I have assembled a scientific advisory board with experts from the areas of political science, sociology, psychology and ecology to have regular discussions on the project status and evolution. These are local experts from my affiliated institutions with whom I have already discussed this proposal and have confirmed their interest in joining the advisory board. Also, a professional advisory board with participants with different roles in relevant OSS projects will be constituted with over 20 volunteers recruited already. Beyond monitoring the evolution of the project and giving their opinion on it, their mission will be to validate and apply on their projects the outcomes of CODE.
The total budget requested is 1.599.697,53€, covering the hiring of 3 postdocs and 3 PhD students (mixing computer science and social science profiles in both categories) and 2 technicians for the duration of the project plus funding for research stays, trips for presenting results, event organizations and equipment.
 This is also a key principle of agile methodologies that have been massively adopted by software teams in the last years but at a small scale.
 We are not implying that all OSS projects should be democratic but we strongly believe that this is an aspect that deserves attention.
 A fork happens when a group of developers take a copy of the source code of a project and use it to create an independent version of the original project, evolving independently (and therefore at the risk of causing a split in the community behind the project if not merged back later on).
 Even if, for whatever reason, a certain project is NOT looking for contributors, stating this clearly (transparency) would avoid misunderstandings.
 A domain-specific language (DSL) is a language specifically designed to express solutions to problems in a specific domain. This is in contrast with general languages (like Java or UML) that aim to be used in any domain.
 Number of citations, h-index and i10 index data taken from Google Scholar. Citations include self-citations.
 R. Schuwer, M. van Genuchten, and L. Hatton, “On the Impact of Being Open,” IEEE Software, vol. 32, no. 5, pp. 81–83, Sep. 2015.
 A. Sutcliffe and N. Mehandjiev, “End-user development: tools that empower users to create their own software solutions – Special issue,” Communications of the ACM, vol. 47, no. 9, p. 31, Sep. 2004.
 F. P. . J. Brooks, “No Silver Bullet Essence and Accidents of Software Engineering,” Computer, vol. 20, no. 4, pp. 10–19, Apr. 1987.
 J. Zhu, M. Zhou, and A. Mockus, “The Relationship Between Folder Use and the Number of Forks : A Case Study on Github Repositories,” in 2014 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, 2011, p. 30.
 B. Vasilescu, V. Filkov, and A. Serebrenik, “Perceptions of Diversity on GitHub : A User Survey,” CHASE Workshop, 2015.
 E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian, “An in-depth study of the promises and perils of mining GitHub,” Empirical Software Engineering, Sep. 2015.
 J. Marlow, L. Dabbish, and J. Herbsleb, “Impression Formation in Online Peer Production : Activity Traces and Personal Profiles in GitHub,” in 16th ACM Conference on Computer Supported Cooperative Work, 2013, pp. 117–128.
 J. Marlow and L. Dabbish, “Activity traces and signals in software developer recruitment and hiring,” in 16th ACM Conference on Computer Supported Cooperative Work, 2013, pp. 145–156.
 F. Fagerholm, A. Sanchez Guinea, J. Borenstein, and J. Munch, “Onboarding in Open Source Projects,” IEEE Software, vol. 31, no. 6, pp. 54–61, Nov. 2014.
 J. Jiang, L. Zhang, and L. Li, “Understanding project dissemination on a social coding site,” in 20th Working Conference on Reverse Engineering, 2013, pp. 132–141.
 K. Apt, Principles of Constraint Programming. Cambridge University Press, 2003.
 L. Ceriani and P. Verme, “The origins of the Gini index: extracts from Variabilità e Mutabilità (1912) by Corrado Gini,” The Journal of Economic Inequality, vol. 10, no. 3, pp. 421–443, Jun. 2011.
 D. Friess and C. Eilders, “A model for assessing online deliberation. Towards a more complex approach to measure and explain deliberativeness online,” in The Internet, Policy & Politics Conferences, 2014.
 L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb, “Social coding in github: transparency and collaboration in an open software repository,” in 15th ACM Conference on Computer Supported Cooperative Work, 2012, pp. 1277–1286.
 R. Padhye, S. Mani, and V. S. Sinha, “A study of external community contribution to open-source projects on GitHub,” in Proceedings of the 11th Working Conference on Mining Software Repositories – MSR 2014, 2014, pp. 332–335.
 J. L. Cánovas Izquierdo, V. Cosentino, and J. Cabot, “Popularity will NOT bring more contributions to your OSS project,” Journal of Object Technology, vol. 14, no. 4, 2015.
 A. van Lamsweerde, “Goal-oriented requirements engineering: a guided tour,” in 5th IEEE International Symposium on Requirements Engineering, 2001, pp. 249–262.
 A. E. Roth, Who Gets What — and Why: The New Economics of Matchmaking and Market Design. Eamon Dolan/Houghton Mifflin Harcourt, 2015.
 A. Bozzon, M. Brambilla, S. Ceri, M. Silvestri, and G. Vesci, “Choosing the right crowd,” in Proceedings of the 16th International Conference on Extending Database Technology – EDBT ’13, 2013, pp. 637–648.
 F. Wiedemann, R. Sontag, and M. Gaedke, “NeLMeS: Finding the Best Based on the People Available Leveraging the Crowd,” in 15th International Conference on Web Engineering, 2015, vol. 9114, pp. 687–690.
 B. Vasilescu, V. Filkov, and A. Serebrenik, “StackOverflow and GitHub: Associations between Software Development and Crowdsourced Knowledge,” in 2013 International Conference on Social Computing, 2013, pp. 188–195.
 L. Singer, F. Figueira Filho, and M.-A. Storey, “Software engineering at the speed of light: how developers stay current using twitter,” in 36th International Conference on Software Engineering, 2014, pp. 211–221.
 D. N. Beede, T. A. Julian, D. Langdon, G. McKittrick, B. Khan, and M. E. Doms, “Women in STEM: A Gender Gap to Innovation,” Economics and Statistics Administration, no. Issue Brief No. 04–11, Aug. 2011.
 V. Cosentino, J. L. C. Izquierdo, and J. Cabot, “Assessing the bus factor of Git repositories,” in 22nd IEEE International Conference on Software Analysis, Evolution, and Reengineering, 2015, pp. 499–503.
 T. Mens and P. Grosjean, “The Ecology of Software Ecosystems,” Computer, vol. 48, no. 10, pp. 85–87, Oct. 2015.
 K. Apt, “Principles of Constraint Programming,” Sep. 2003.
 “Dear GitHub – An open letter from the maintainers of open source projects.” [Online]. Available: https://github.com/dear-github/dear-github.
 H. C. Esfahani, J. Cabot, and E. Yu, “Adopting agile methods: Can goal-oriented social modeling help?,” Research Challenges in Information Science (RCIS), 2010 Fourth International Conference on, 2010.
 J. Cabot, R. Clarisó, and D. Riera, “On the verification of UML/OCL class diagrams using constraint programming,” Journal of Systems and Software, vol. 93, pp. 1–23, Jul. 2014.
 C. A. González and J. Cabot, “Formal verification of static software models in MDE: A systematic review,” Information and Software Technology, vol. 56, no. 8, pp. 821–838, Aug. 2014.
 V. Cosentino, J. L. C. Izquierdo, and J. Cabot, “Assessing the bus factor of Git repositories,” in 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), 2015, pp. 499–503.
 R. Tairas and J. Cabot, “Corpus-based analysis of domain-specific languages,” Software & Systems Modeling, vol. 14, no. 2, pp. 889–904, Jun. 2013.
 D. Ameller, C. Ayala, J. Cabot, and X. Franch, “Non-functional Requirements in Architectural Decision Making,” IEEE Software, vol. 30, no. 2, pp. 61–67, Mar. 2013.
 K.-J. Stol and B. Fitzgerald, “Inner Source–Adopting Open Source Development Practices in Organizations: A Tutorial,” IEEE Software, vol. 32, no. 4, pp. 60–67, Jul. 2015.
 M. Nagappan, T. Zimmermann, and C. Bird, “Diversity in software engineering research,” in Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, 2013, p. 466.
 A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design science in information systems research,” MIS Quarterly, vol. 28, no. 1, pp. 75–105, Mar. 2004.
 C. Larman and V. R. Basili, “Iterative and incremental developments. a brief history,” Computer, vol. 36, no. 6, pp. 47–56, Jun. 2003.

References: V. 
 V. 
 V. 
 V. 
 V. 
 V. 
 V.