Using Web Scraping and Network Analysis to Study  International Collaboration in Open Source Software

Brandon L. Kramer1 Gizem Korkmaz1

J. Bayoán Santiago Calderón2	Carol A. Robbins3

1

2

3

Federal Committee on Statistical Methodology 

 2021 Research and Policy Conference

This work is supported by the National Center for Science and Engineering Statistics (49100420C0015).

The views expressed in this work are those of the authors and not necessarily those of their respective institutions.

1 / 23

ProjectBackground

Presentation Overview

Project Background  

	Open Source Software  International Collaboration

Data & Methods

Data Collection & Classification  Networks Construction

OSS International Collaboration

Contributor networks

Country-to-country networks

2 / 23

ProjectBackground

What is Open Source Software?

Software that is published under an OpenSourceInitiative(OSI)  approved license

OSI-approved licenses establish permissions (e.g., use, inspect,  modify, distribute, attribution) and limitations (e.g., liability, warranty)  Most common licenses are: MIT, Apache, GPL, BSD, etc.

Prominent OSS examples include: Apache, Linux, Mozilla, R, etc.  Past work has conducted network analysis of either single projects  and/or smaller-scale networks of code hosting platforms like GitHub

3 / 23

ProjectBackground

The Scope and Impact of OSS

Current NCSES and other economic indicators do not measure  the scope and impact of open source software developed  outside the business sector.1

Scope and Value: How much open source software is in use?  Who creates these products? How can we measure the value of  open source software?

Collaboration Networks: What is the structure of OSS collaboration  networks? How do collaborations span across geographic boundaries?

1S. Keller, G. Korkmaz, et al. “Opportunities to observe and measure intangible inputs to innovation:  Definitions, operationalization, and examples”. In: Proceedings of the National Academy of Sciences  115.50 (2018), pp. 12638–12645.

4 / 23

ProjectBackground

International Collaboration

International collaboration doubled in academic papers since 1990

More governmental funding for projects that developed through  international collaboration tend to lead to higher impact publications

Understanding international contributions in the context of OSS will  help explain:

Which countries are most likely to contribute to OSS?  Which countries are most likely to collaborate internationally?

What is the structure of international collaboration and  how does it change over time?

Which are the most influential countries in the OSS ecosystem?

Using network analysis to study:

Individual and country-to-country collaborations

5 / 23

Data&Methods

Data Collection

Developed GHOST.jl to scrape commit data

Package developed for targeted scraping of GitHub user and activity  data using the GitHub v4 GraphQLAPI

Find public repositories with an OSI-approved license

Collect development activity information (e.g., commits, additions)

Used GHTorrent to classify contributors

Commit data supplemented with user data from GHTorrent2  User data includes login, email, location and company information  Developed algorithm to convert location data to country codes

2Georgios Gousios. “The GHTorrent dataset and tool suite”. In: Proceedings of the 10th Working  Conference on Mining Software Repositories. MSR ’13. San Francisco, CA, USA: IEEE Press, 2013,  pp. 233–236.

6 / 23

Data&Methods

Tools for Scraping OSS Data

GHOST.jl*

Julia package used for targeted scraping of commit activity data

tidyorgs*

R package used for organizational and sectoring classification

diverstidy*

R package used for geographic and population classification

PyGithub

Python package used for scraping user and repository attributes

* Developed at UVA SDAD

7 / 23

Data&Methods

Data Summary: Contributors

Our original dataset is comprised of 3.3M distinct contributors and

7.8M distinct repositories

To examine international collaborations, we reduced the dataset to only  include logins with valid country codes, which included 733K  contributors and 3.5M repositories dating from 2008 to 2019

8 / 23

Data&Methods

Network Data

To convert these data into network format, we “projected” a bipartite  login-repository network into single-mode contributor networks

The contributor network is comprised of

Nodes represent contributors

Edges correspond common repositories that users contribute

The country-country network is comprised of

Nodes represent countries

Edges correspond to international collaborations

Analyzed networks using R’s igraph and ggraph as well as Gephi

9 / 23

Results

International Collaboration Tendencies

US engages in domestic collaboration more than other top countries  Top countries collaborate with US developers more than domestic colleagues

10 / 23

Results

Longitudinal Trends

Marked contrast with exponential growth of contributor networks  Countries join the network until around 2013

Collaborations steadily rise while commits increase exponentially

The number of communities fluctuate in a way that  clearly depends on the upper threshold of countries

11 / 23

Results

Longitudinal Trends

The density, transitivity, modularity, and mean distance  all reflect a similar shift around 2013

12 / 23

Results

Community Detection Analyses

13 / 23

Results

Community Detection Analyses

The inclusion of domestic collaborations (loops) in our community  detection analyses revealed regional collaboration tendencies

14 / 23

Results

Community Detection Analyses

Regional communities formed between Nordic,  African, South American, and former-Soviet countries

15 / 23

Results

Comparing Centrality Measures

Comparing betweenness centrality vs. degree centrality shows more relative  brokering capacity for the US, Canada, China, Nigeria and Kenya

16 / 23

Results

Comparing Centrality Measures

Examining average betweenness centrality over time reveals  that top countries have started yielding influence

17 / 23

MainTakeaways

Main Takeaways

GitHub Contributor Networks

US-based users are most likely to contribute to GitHub

Country-to-Country Networks

Network growth affected by upperbound of countries in the world

Centrality measures reveal that some countries (like the USA) have  more influence in the OSS ecosystem

The country-country network clusters into regional communities,  reflecting a likely combination of socio-political and economic  factors that shape OSS development

Policy Implications

Use of SNA can help the federal government monitor the level of  incoming and outgoing OSS projects as well as their economic value  by capturing international collaboration dynamics over time.

18 / 23

MainTakeaways

Q&A

Questions?

19 / 23

MainTakeaways

Summary of Scraped GitHub Data

20 / 23

MainTakeaways

Summary of Scraped GitHub Data

21 / 23

MainTakeaways

Summary of Results

 Country Users  Repos  R/U  Commits C/R   Adds	A/C   Dels  D/C  TotColabs DomColabs USColabs
| USA     | 216K | 1.2M | 5.3 | 43.0M | 37.5 | 65.9B | 1531.6 | 26.7B | 621.5 | 2.4M | 26.4% | 26.4% |
|---------|------|------|-----|-------|------|-------|--------|-------|-------|------|-------|-------|
| China   | 54K  | 289K | 5.3 | 6.6M  | 22.9 | 13.7B | 2080.1 | 4.9B  | 633.5 | 339K | 14.9% | 33.2% |
| Germany | 40K  | 298K | 7.4 | 11.4M | 38.2 | 11.2B | 986.8  | 5.2B  | 452.8 | 790K | 9.0%  | 30.1% |
| UK      | 40K  | 271K | 6.8 | 9.4M  | 34.8 | 11.2B | 1191.6 | 5.2B  | 550.0 | 548K | 8.2%  | 32.6% |
| India   | 37K  | 160K | 4.3 | 2.7M  | 16.9 | 7.1B  | 2615.7 | 1.9B  | 684.5 | 306K | 6.9%  | 33.2% |
| Canada  | 27K  | 174K | 6.4 | 5.1M  | 29.4 | 7.4B  | 1449.9 | 3.0B  | 585.9 | 335K | 4.5%  | 33.9% |
| Brazil  | 25K  | 142K | 5.7 | 2.8M  | 20.0 | 6.0B  | 2115.4 | 2.1B  | 749.5 | 165K | 6.2%  | 29.0% |
| France  | 25K  | 173K | 7.0 | 6.0M  | 34.7 | 7.8B  | 1287.2 | 3.0B  | 499.2 | 375K | 6.4%  | 28.0% |
| Russia  | 22K  | 120K | 5.5 | 3.4M  | 28.5 | 3.6B  | 1063.5 | 1.5B  | 435.6 | 197K | 5.9%  | 27.6% |
| Japan   | 16K  | 130K | 8.2 | 3.9M  | 29.9 | 3.7B  | 955.9  | 1.5B  | 390.4 | 205K | 6.7%  | 32.6% |

Descriptive Statistics for Top-10 Countries Based on Users, GitHub 2008-19

The table shows US-based contributors higher number of users, repos,  commits, and overall collaborations relative to other countries in the  network. We also observed substantial country-level variability in the  number of repos per user (R/U), the commits per repo (C/R),

as well as the additions (A/C) and deletions per commit (D/C).

22 / 23

MainTakeaways

Summary of Results

 Year	Nodes  Edges  Commits  Triads  Cmtys  Comps  K -Core  Dens  Trans  Mod   μDist μDeg μBtw   GCent GBtw
| 2008    | 91  | 1183  | 123K  | 11K  | 36 | 15 | 33 | 0.289 | 0.725 | 0.001 | 1.669 | 24.0 | 22.1 | 0.533 | 0.170 |
|---------|-----|-------|-------|------|----|----|----|-------|-------|-------|-------|------|------|-------|-------|
| 2008-09 | 106 | 1713  | 320K  | 20K  | 35 | 18 | 38 | 0.308 | 0.742 | 0.003 | 1.624 | 30.3 | 23.6 | 0.511 | 0.125 |
| 2008-10 | 128 | 2220  | 688K  | 30K  | 41 | 18 | 45 | 0.273 | 0.734 | 0.003 | 1.703 | 32.7 | 34.1 | 0.546 | 0.117 |
| 2008-11 | 153 | 3028  | 1.3M  | 45K  | 38 | 15 | 51 | 0.260 | 0.696 | 0.002 | 1.733 | 37.6 | 46.6 | 0.601 | 0.159 |
| 2008-12 | 177 | 3939  | 2.3M  | 64K  | 35 | 13 | 55 | 0.253 | 0.657 | 0.002 | 1.755 | 42.5 | 58.4 | 0.633 | 0.142 |
| 2008-13 | 208 | 4944  | 4.0M  | 87K  | 50 | 22 | 59 | 0.230 | 0.633 | 0.002 | 1.763 | 45.5 | 64.5 | 0.611 | 0.101 |
| 2008-14 | 219 | 5992  | 6.5M  | 120K | 46 | 20 | 63 | 0.251 | 0.646 | 0.001 | 1.748 | 52.7 | 68.7 | 0.616 | 0.107 |
| 2008-15 | 230 | 7160  | 10.5M | 161K | 52 | 24 | 70 | 0.272 | 0.657 | 0.001 | 1.695 | 60.3 | 65.1 | 0.601 | 0.087 |
| 2008-16 | 236 | 8177  | 15.8M | 199K | 53 | 22 | 74 | 0.295 | 0.657 | 0.000 | 1.665 | 67.3 | 64.9 | 0.594 | 0.060 |
| 2008-17 | 240 | 9270  | 22.0M | 245K | 52 | 18 | 81 | 0.323 | 0.667 | 0.001 | 1.653 | 75.2 | 68.0 | 0.572 | 0.052 |
| 2008-18 | 241 | 10565 | 28.9M | 314K | 48 | 12 | 90 | 0.365 | 0.697 | 0.003 | 1.619 | 85.7 | 67.7 | 0.539 | 0.042 |
| 2008-19 | 241 | 11294 | 34.7M | 348K | 45 | 18 | 93 | 0.391 | 0.696 | 0.002 | 1.586 | 91.7 | 64.0 | 0.534 | 0.035 |

Longitudinal Descriptive Analysis of Country Networks, GitHub 2008-19  (Cmtys=Communities, Comps=Components, Btw=Betweenness)

At the country-level, the network moves through two growth periods marked  first by rapid growth before all countries join the network. After 2013, the  network becomes more dense and more transitive as the number of distinct  communities drops and communities of OSS collaboration form around  specific geographical regions.

23 / 23

# image1.png

# image2.png

# image3.png

# image4.jpg

# image5.jpg

# image6.png

# image7.png

# image8.png

# image9.png

# image10.png

# image11.png

# image12.png

# image13.png

# image14.png

# image15.png

# image16.png

# image17.png

# image18.png

# image19.png

# image20.png

# image21.png

# image22.png

# image23.png

# image24.png

# image25.png

# image26.jpg

# image27.png

# image28.png

# image29.png

# image30.png

# image31.png