On Updating Library Dependencies inOpen Source Software Projects

Raula Gaikovina Kula, Dr. Eng
Software Engineering Lab (Matsumoto ken)Special International Track on
Software Analytics (2) (Invited Talk)
Get the Slides
https://goo.gl/6UcAHCAbout MeMSR

2

Outline of my Talk
1.
2.

3.

4.

Background: Third-party Libraries (5 mins)○ Rise of Library Ecosystems
Motivation : What about a Library Update?(10 mins)○ Library Updates in Practice○ Awareness Mechanisms
Main Study: A Study on Library Updates (10 mins)○ Library Updates in Practice○ Awareness Mechanisms
Future and Challenges: Library Ecosystems in the Future (5 mins)○ Ongoing work & Software Analytics

3

Imagine a Developer in Need of afunction, feature ...

4

Libraries to the rescue© https://www.youtube.com/watch?v=aIB5xohKUZI5

1. Background - Third-Party Library Usage
Modern Landscape of Software Development isbuilt on Third Party Libraries-Suppose you want to write some unit tests
in Java… just adopt the junit library© Images taken from respective software repositories

6Library Dependencies7© https://www.tes.com/lessons/vfqedGrGvRj7dw/ecosystems8

Results in Software (Library) Ecosystems

https://search.maven.org/

https://www.cpan.org/

https://rubygems.org/

https://cran.r-project.org/

475,000 building blocks
https://www.npmjs.com/

9© https://www.tes.com/lessons/vfqedGrGvRj7dw/ecosystems
10What is a Software Ecosystem?
Adapted from biological ecosystems:
Scypersrsky [1]: “defined as a set of businesses functioning as a unit andinteracting with a shared market for software and services, together withrelationships among them.”
Lungu [2]: “a collection of software systems, which are developed andco-evolve in the same environment”
Stallman [3] : “It is a mistake to describe the free software community, or
any human community, as an "ecosystem", because that word implies theabsence of (1) intention and (2) ethics"11© https://www.scmagazineuk.com/researchers-developing-cyber-attack-predictive-model/article/540346/

12

2. Motivation
Updates are sometimes strong recommended,
especially with Security Vulnerabilities

CVE-2014-6271© https://en.wikipedia.org/wiki/Shellshock_(software_bug)

CVE-2014-3566

CVE-2014-0160© https://en.wikipedia.org/wiki/POODLE© http://heartbleed.com/

13

Update Awareness: Security Advisories

Example:https://cve.mitre.org/cgi-bin/cvename.cgi?name=cve-2014-0160© https://www.singleblackmale.org/2010/02/17/spider-sense/

https://nvd.nist.gov/
14

Relentless mining of data ….
MSR Icon© https://www.ausimmbulletin.com/feature/innovation-in-mining/https://search.maven.org/

15

Systematic Modeling of System & Library Usage

Library migration between systems and libraries. Theorange arrow depicts dependency relations between
them.

16

Library Tracking ModelSimple example of the LU-basedmetrics.
We show the Peak LU at time t1,current LU at time t2 and library
residue (Peak LU / Current LU).

17

Visualizing Library Usage

A Library Migration Plot. In this example, therelease of a related security advisory
CVE-2014-0114 (black dashed line) that affects
beanutils versions 1.9.1 (marked with crossbones).
18

3. An Empirical Study on Library Updates
Motivation: In 2014, Sonatype determined that over 6% of thedownload requests from the Maven Central repository included
known vulnerabilities.
Related Work at the API level:●●●Robbes et al. [4] - Smalltalk ecosystem
Hora et al. [5] - Pharo
Sawant et. al [6] - Java.
○Bavota et al. [7] - Apache products.

We study both library and system perspectives, as well as theawareness and the nature (new release or security vulnerability).

19

Library Migration in Practice
RQ1: To what extent are developers updating their library dependencies?

★
★

4,659 GitHub Projects
2,700 Java Maven libraries“We find that (i) although system heavily depend on libraries,
most systems rarely update their libraries and (ii) systems areless likely migrate their library dependencies, with 81.5% ofsystems remaining with a popular older version”.

20Effectiveness of Awareness Mechanisms (1/2)
RQ2: To what extent are developers updating their library dependencies?
★
★

★

★

3 new releases of popular libraries
5 security vulnerabilities“New release of a popular library (i) there exist patterns of consistentmigration and patterns where an older popular library version is still
preferred.”©
https://www.singleblackmale.org/
2010/02/17/spider-sense/“For a security advisory disclosure we find cases of developer (ii) nonresponsiveness to security advisory disclosure, which is sometimes
due to an incomplete patch or a latent security advisory.”
21

Effectiveness of Security Advisory (2/2)
RQ3: Why are developers non responsive to a security advisory?

★
★

16 Projects contacted for feedback (detected as vunerable in RQ2)
Understand feedback“69% of developers were unaware of their vulnerable
dependencies and proceeded to immediately migrate to asafer dependency.”★
★

Developers evaluate based on project specificpriorities.
Developers cite migration as a practice that requires
extra migration effort and added responsibility.
22

The Complex Web of Updates &Library Ecosystems-We find that developers are wary of the latest versions© https://www.boredpanda.com/old-superhero-paintings-eddie-liu/

23

The Complex Web of Updates &Library Ecosystems-We find that developers are wary of the latest versions
Leads to Dependency Hell with updates in the software
ecosystem...

244. Future & Challenges
Perfect World: Real-time Updates, no breakages
and the ecosystem is robust to handle anychanges:

Reality: Work in Progress...
Challenges at the Software Ecosystem Level...

25

Library Recommendations and Visualizations
(1\2)

26

Library Recommendations and Visualizations
(2\2)

27

Recent Ideas●Studying Security
Vulnerabilities inOther Ecosystems(npm)
https://snyk.io/28

Recap and thank you...
★

★

★

★

Background: Third-party Libraries○ Rise of Library Ecosystems○ In Libraries we Trust
Motivation : What about a Library Update?
○ Library Updates in Practice○ Awareness Mechanisms
Main Study: A Study on Library Updates○ Library Updates in Practice○ Awareness Mechanisms
Future and Challenges: Library Ecosystems in the Future○ Ongoing work
○ Role of Software Analytics○Systematic Modelling29