Vulnerability Reachability Analysis Using OSS
Tools___________________________
Mike Larkin
Deepfactor, Inc.
mlarkin@deepfactor.io# #For# more AppSec

information and resources:- Visit deepfactor.io- Sign up for free trial of
Deepfactor Application
Security

// Agenda> Overview (~20 minutes)> Types of reachability analysis (~10 minutes)> Call graph analysis exercise (~10 minutes)> Dynamic/runtime analysis exercise (~10 minutes)> Results comparison (~10 minutes)> Conclusion / Q&A (~10 minutes)

Overview# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
3

// About Me> Co-founder and CTO at Deepfactor• We make software to help people prioritize vulnerability remediation> Adjunct faculty at San Jose State University• Computer Engineering (CMPE) MS degree program• Virtualization Technologies, Software Security, and Operating Systems> Active open-source contributor• OpenBSD (hypervisor, device drivers, memory/device management, ACPI)DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

4

// Goals Of This Workshop> This workshop has several goals; at the end of the workshop, you should • Know what reachability analysis is, and why you should care about it• Know why reachability can help you prioritize vulnerability remediation• Understand the different types of reachability analysis tools• Learn where you can reach out for help in this area later

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

5

// If You Want To Follow Along …
> I will be doing 3 examples today that you can also do yourself• … if you want. Otherwise, sit back and relax and enjoy the beer and food

> The list of what you will need to install is pretty simple:• Trivy• Go

https://trivy.dev
https://go.dev• Java• Gradle

https://openjdk.org
https://gradle.org (if you want to try the Java example)> For the Go example, you’ll need some Go app (of your choice)• I’m going to demo JIRA-CLI : https://github.com/ankitpokhrel/jira-cli
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

6

// Vulnerability Reachability Analysis> Code that is contains vulnerabilities is bad> Code that contains vulnerabilities used in your application is worse> How do you know if some code you are using is vulnerable?
> Better yet, how do you know you’re even using the vulnerable code at all?

> These questions are what we are going to focus on today

// Vulnerability Reachability Analysis> We will start by talking about reachability> We’ll then talk about what vulnerabilities are, and how they are managed> Then we will look at tools you can use to catalog what CVEs you might have inyour code> Finally, we’ll conclude with some short examples with open source tools to doyour own reachability analysis

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

8

Reachability# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
9// Reachable Code> How do you define reachability?
> Certainly, code that your program executes is, by definition, reachable> What about code that is packaged with your program but never loaded?
> What about code that is loaded by your program but never executed?
> What about code sitting on the same machine/container that could theoreticallybe launched?

// Reachable Code> “Code that is packaged with your program but never loaded”> I’d suggest getting rid of that code
> There are tools to help you locate such code

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

11

// Reachable Code> “Code that is loaded by your program but never executed”> For example• Shared library dependencies created by the linker but not used
• Java apps doing Class.forName(…) but never using any methods in the class• dlopen(…) but never using the thing you loaded> This might happen in applications that support things like plugins, but then theloaded module isn’t ever exercised> Code like this is reachable!
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

12

// Reachable Code> “Code sitting on the same machine that might be launched”> Out of scope (for this talk…)> This is sort of like the earlier example though; if it’s not used, why is it there?
> No need to leave lolbins laying around for an
attacker

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

13

// Reachable Code> If we distill the previous scenarios down to the two important ones …
• Code directly executed by your program• Code loaded into the address space/interpreter by your program (maybe used, maybenot)> How do you know which functions/methods fall into each category?
> Said a different way, how can you compile a definitive list of functions andmethods that are reachable, according to the previous definitions?

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

14

// Reachability Analysis> Before we discuss “how”, let’s talk about “why”> Why is creating this list important?
> Simple answer –Reachable code that contains vulnerabilities should be remediated with priority

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

15

// Reachability Analysis> If you have several vulnerabilities to fix…
• Prioritize fixing the ones that are reachable, with known exploit PoCs first• Next focus on the other reachable ones
• Then focus on the rest, based on severity> All that advice depends on knowing what is reachable

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

16

// Reachability Analysis> There are generally two types of reachability analysis tools• Tools that scan source code and generate a call graph based on syntax analysis> foo().bar().baz()-> “methods foo, bar, baz are reachable”• Tools that monitor the program after it is built, and watch what is loaded or executed> Profiling, library call interception, etc> Each of these approaches can produce a list of reachable functions/methods> Each approach has strengths and weaknesses

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

17

Vulnerabilities &Bad Code# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
18

// Let’s Talk About Software Vulnerabilities> Vulnerable code is everywhere.
• You’re using it• I’m using it• Even my dog is using it> Let’s talk about where vulnerable code comes from

// Let’s Talk About Software Vulnerabilities> What causes a vulnerability?
• Are vulnerabilities caused by incorrect (buggy) code?
• Is correct code vulnerability free?
• Is vulnerability free code always correct?
• Are vulnerabilities in your program always the result of code you wrote?

// Stupid Example

What do we think about this?
Is this correct (bug free)?
Could this code have a vulnerability?

int data[MAX_DATA];/* Return data at position “index” */intfunction(int index){
int i;
i = data[index];
return i;}

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

21

// Stupid Example #2

What do we think about this?
Is this correct (bug free)?
Could this code have a vulnerability?

int data[MAX_DATA];/* Return data at position “index” */intfunction(int index){
int i = -1;
if (index < MAX_DATA)i = data[index];
return i;}

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

22

// Stupid Example #3

What do we think about this?
Is this correct (bug free)?
Could this code have a vulnerability?

int *numbers;/** set numbers[x] = x for all x > 0 and < size .
*/voidfunction(int size){
int i;
numbers = malloc(size * sizeof(int));
for (i = 0; i < size; i++)numbers[i] = i;}

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

23

// Yes, Those Were Stupid> Type confusion• Misunderstanding the meaning of a value> Corner cases• Not checking for all error conditions> Not checking return values> Undefined behavior• Of course, nobody here would ever make such
mistakes…
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

24

// Not So Obvious Example

static unsigned int
tun_chr_poll(struct file *file, poll_table * wait){
struct tun_file *tfile = file->private_data;
struct tun_struct *tun = __tun_get(tfile);
struct sock *sk = tun->sk;
unsigned int mask = 0;
if (!tun)return POLLERR;. . .
}https://lwn.net/Articles/342330
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

25

// Not So Obvious Example

static unsigned int
tun_chr_poll(struct file *file, poll_table * wait){
struct tun_file *tfile = file->private_data;
struct tun_struct *tun = __tun_get(tfile);
struct sock *sk = tun->sk;
unsigned int mask = 0;
if (!tun)return POLLERR;. . .
}https://lwn.net/Articles/342330
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

26

// Why Did We Look At These Stupid Examples?
> These examples were shown to illustrate a few points• Even simple mistakes or accidents can cause a vulnerability• Vulnerabilities are everywhere• They are not going away
• You probably didn’t write the bad code yourself• We need a way to track them and prioritize
remediation
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

27

// Bad Code Is Out There> We’ll never be able to get rid of bad code
> Mistakes, laziness, apathy, and inexperience can all contribute to the problem• (Ehm, memory unsafe languages, too)

> Even if you write 100% perfect bug-free, vulnerability-free code, you are still
likely to step on landmines• Importing third party code/dependencies• Downstream refactoring• Code being used in unexpected ways

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

28

// CVEs> Known vulnerabilities can be assigned a CVE number for tracking• Each CVE is assigned a severity• Each CVE can contain information about the vulnerability• Each CVE can contain information about “fixed-in” versions• … plus arbitrarily more information …

> Who assigns CVEs?
> What are they used for?
> Who decides the severity and other information included in the report?
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

29

// Example Of A Meaningless CVE

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

30

// Better Example

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

31

// CVEs (cont’d)> How do you know if you’re vulnerable to a CVE?
> To answer the question, it’s important to know what components you are using
in your application• After all, if you aren’t using component XYZ at all, then you’re certain to not be subject
to any of its vulnerabilities> Ok, so how do you know what components you are using in your application?

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

32

// Imports> If you’re lucky, your language orcompiler might tell you• For example, go.mod> The developer might also tell you• Gradle or .pom files• package_lock.json> Or maybe you can scan your program
and try determine what it uses, if you
don’t know

require (github.com/AlecAivazis/survey/v2 v2.3.7
github.com/atotto/clipboard v0.1.4
github.com/briandowns/spinner v1.23.0
github.com/charmbracelet/glamour v0.6.0
github.com/cli/safeexec v1.0.1
github.com/fatih/color v1.15.0
github.com/gdamore/tcell/v2 v2.6.0
github.com/google/shlex v0.0.0-20191202
github.com/kballard/go-shellquote v0.0.0-201851
github.com/kentaro-m/blackfriday-confluence v0.0.0-2022
github.com/kr/text v0.2.0
github.com/mattn/go-isatty v0.0.19. . .
github.com/alecthomas/chroma v0.10.0 // indirect
github.com/alessio/shellescape v1.4.1 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/creack/pty v1.1.18 // indirect
github.com/danieljoos/wincred v1.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dlclark/regexp2 v1.10.0 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect. . .
)DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

33

// Example - Trivy> Trivy can be used to scan a program’s dependencies• Plus container images, filesystems, etc• https://github.com/aquasecurity/trivy> Let’s scan a container

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

34

// SBOMs> Software Bill Of Materials• Similar to a BOM for a physical thing like a car, toaster, or television• Lists all the things required to build the “thing” (software in this case)> Instead of nuts, bolts, flanges, and circuit boards, we have lists of software packages and theirversions• Can be described in various formats (SPDX, CycloneDX)> Biden executive order 14028• https://www.ntia.gov/sites/default/files/publications/sbom_myths_vs_facts_nov2021_0.pdf

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

35

// SBOMs (cont’d)> With an SBOM, an organization is empowered to …
• Answer the question “Am I affected?” more easily when a vulnerability is discovered> Minutes or hours, not days or weeks later• Determine which components are affected• Determine roadmaps for remediation, when coupled with reachability insights

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

36

// SBOMs (cont’d)> SBOM content can be correlated with CVE databases> This would give you a list containing two things• Components used to build your application• Vulnerabilities present in those components> Surely that be enough to prioritize what gets fixed first, right?

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

37

// Sample SBOMDEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

38

// Sample SBOM

Yikes!
784 Vulnerable
Components?!?

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

39

// Tidal Wave
> That’s not solvable> You’re going to get crushed by the neverending wave of CVEs

> Let’s fix the problem

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

40

Call Graph
Reachability
Analysis# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
41

// Call Graphs> There are tools that can tell you what code is reachable in your application atbuild time
> These tools scan your source code and produce a graph of “what calls what”> That graph is then traversed to create a list of reachable code paths> The hope here is that by knowing what is possibly callable, we can define the listof reachable code
> With that information in hand, we should be able to prioritize remediation tasks

// Call Graphs> A compiler analyzes your code during buildand creates a syntax treefoo(){bar();> Some nodes in this tree can be call sites(locations where program flow transitions
from one function to another)

}
bar()
{
baz();> Call sites can be cataloged to create the“what calls what” list

}
baz()
{
. . .
}

// Call Graphs> In this example, we know that foo calls bar
and bar calls baz

foo(){bar();> Assuming foo is called from somewhere
else, then our list of reachable code
consists of
• foo

}
bar()
{
baz();

• bar

}

• baz

baz()
{
. . .
}

// Call Graphs> This list can help us prioritize remediating
any CVE that includes one of thesefunctionsfoo(){bar();}• Eg, “A remote code execution vulnerability
exists in libFooBarBaz.so if the baz() function
is called.”bar(){
baz();
}
baz()
{
. . .
}

// Call Graphs> A different example> What can we say about the reachability of“hamburger”?
> It’s not called from anywherefoo(){bar();}
bar(){
printf(“hello”);}> Is it reachable?

hamburger()
{
. . .
}

// Call Graphs> Language complexities make it difficult tocatch all the casesfoo(){
if (some_param) == 42
ptr = baz;else
ptr = bar;

• Function pointers• Reflection based invocation• Function names not known at compile time

ptr();}> Is bar() called here? What about baz()?
> Are either of them or both reachable?

bar()
{
. . .
}
baz()
{
. . .
}

// Call Graphs> What can we say about thereachability of various code here?
> Is any code even executed from class“name” in this example?

public void myMethod(String name){Object o =class.forName(name);. . .
}> It’s difficult to get a complete pictureof what’s going if all you have to lookat is the source

go install github.com/ofabry/go-callvis@latest// Call Graph Example> I’ll be showing how to produce a call graph from a Go application using‘callgraph’• https://github.com/golang/tools/tree/master> This tool should work against any Go application for which you have source

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

49

Runtime Reachability
Analysis# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
50

// Runtime Reachability> Tools that use runtime reachability analysis create the list of reachable code byexamining the program while it runs
> These tools generally do not look at source code, although they may, foradditional context (eg, if the tool also produces SBOMs)> By monitoring what is used by the program, the list of reachable code can becreated// Runtime Reachability> Since the list of reachable code is defined by what is used during monitoring,care must be taken to ensure the system under test is exercised fully> Tools employing runtime reachability can have different granularities• Function level• Module level> Function level tracing gives more specificity but can produce substantial output> Module level tracing omits some specificity and assumes “module loaded” means“code in that module is reachable”
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

52

// Runtime Reachability> How do these tools work?
> Some intercept library calls to monitor when specific functions are called> Some use traditional profiling techniques (periodic stack sampling)> Some emulate or partially emulate the program’s execution to monitor calls> Each approach is slightly different but all fall under the category of runtime
analysis

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

53

// Runtime Reachability Example> I’ll be showing how to generate a list of called/used Java classes at runtime usinga small bytecode rewriting agent• The agent can be found here:• https://github.com/deepfactor-io/reachability-workshop

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

54

Putting It All
Together
Reachability + Prioitization# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
55

// Recap> Ok, so at this point we’ve done the following• Scanned our application and produced an SBOM
• Using the SBOM, correlated which CVEs we might be vulnerable to based on the SBOM
contents• Performed a reachability analysis exercise on our code which gave us a list of modules
or functions used

> How do we put all this together to arrive at a prioritization order?

// First Step – Code-To-Module> Let’s pretend that we have built a listof reachable code that looks like this
• The list could have been created using
either approach (call graph analysis or
runtime analysis)> What’s next?

/usr/lib/libfoo.so
foo()
bar()
baz()
/usr/lib/libyummy.sohamburger()hotdog()sushi()/usr/bin/myappmain()
func1()
func2()
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

57

// First Step – Code-To-Module> We need to get from this list of
modules to something that matches
what we have in our SBOM
> Remember, CVE lists are often sourced
from the software vendor and thus will
be using vendor package names• Eg, “libyummy-1.2.3p1” not“/usr/lib/libyummy.so”/usr/lib/libfoo.so
foo()
bar()
baz()
/usr/lib/libyummy.sohamburger()hotdog()sushi()/usr/bin/myappmain()
func1()
func2()
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

58

// First Step – Code-To-Module> Assuming you have a package
manager, the reverse file mapping
capability is useful here
• rpm –qf
• dpkg –S• apk info --who-owns• ...

> P.S. This is one reason a package
manager is important …

/usr/lib/libfoo.so
foo()
bar()
baz()
/usr/lib/libyummy.sohamburger()hotdog()sushi()/usr/bin/myappmain()
func1()
func2()
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

59

// Second Step – Module-To-CVE> Now that we have the list of modules,
a query against the list of CVEs weobtained previously can be made• grep, sed, awk, jq, whatever…
• Can add thresholds or ordering in thisstep, based on your organization’s appsec
policies> Note: Your own executable/classprobably won’t be packaged this way/usr/lib/libfoo.so :: libfoo-61.7/usr/lib/libyummy.so :: libyummy-1.3/usr/bin/myapp$ jq ‘.[package]’ cvelist.json | grep libfooCVE-2024-12345:
A vulnerability exists in libfoo’s baz()function …

• And even if it was, it would be you
issuing CVEs for it anyway
DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

60

// Second Step – Module-To-CVE> Sometimes the CVE text will tell you
definitively which function is bad> Most of the time you need to becontent with just assuming if you usedanything in the module that you should
throw it out or upgrade• Vendors are disincentivized to provide
real useful information/usr/lib/libfoo.so :: libfoo-61.7/usr/lib/libyummy.so :: libyummy-1.3/usr/bin/myapp$ jq ‘.[package]’ cvelist.json | grep libfooCVE-2024-12345:
A vulnerability exists in libfoo’s baz()function …

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

61

// Second Step – Module-To-CVE> In the end, we’ve produced thefollowing• A list of CVEs …
• … applicable to modules we have in ourSBOM• … that we provably used code from in our
program/usr/lib/libfoo.so :: libfoo-61.7/usr/lib/libyummy.so :: libyummy-1.3/usr/bin/myapp$ jq ‘.[package]’ cvelist.json | grep libfooCVE-2024-12345:
A vulnerability exists in libfoo’s baz()function …

> Using this approach, we now have a listof the “most important” CVEs

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

62

// Final Step
> Of course, you could take this further• Further refine the list to prioritize CVEs with known public exploits> EPSS score is a way of tracking this• “Is an exploit available?”• “What is the likelihood of an exploit becoming available in the next 30/60/90 days?
• Some tools incorporate EPSS into their severity ranking> VEX enhancements to CVEs
• Sometimes more information can be gleaned

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.

63

Conclusion# # #DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
64

// Conclusion> What have we learned?
• We learned what it means for code to be reachable• We learned why we need to care about vulnerabilities• We learned how to scan our code for CVEs
• We learned how to apply reachability analysis to discover which modules have reachablecode• We learned how to create a prioritized list of CVEs based on reachability

Thank You
For more AppSec
information and resources:- Visit deepfactor.io- Sign up for free trial of
Deepfactor Application
Security

DEEPFACTOR CONFIDENTIAL 2022. ALL RIGHTS RESERVED.
66