What’s In Your AI Code?

Darren Meyer, Sta Research Engineer — 26. June 2024

fffi

"hyperspace sh bone" by jurvetson is licensed under CC BY 2.0.

What do we meanby “AI Code”?

fiYo
urcoolne
wA

Yo
ur
co
den
soiet
ir
ar
alr
uebgidLon
MoCI-enable

dproductHow dependencies
work
Package Managers and Runtimes tend to operatecompletely decoupled, like ships passing in the night
1. Developer starts a new project that uses a couple ofdependenciesreganaM
Package

2. Developer creates a manifest le to declare the two
direct dependencies (requirements.txt, package.json,pom.xml, etc.)
3. Build system runs package manager and the direct
dependencies bring along several other transitive
dependencies
4. Package manager copies the les in a directoryfifi

5. Runtime/compiler loads dependencies as needed
during execution

Compiler / Runtime

Things get out of sync
It is unavoidable• Developers import new dependencies without
updating the manifest le (possible in python,Javascript, scripts etc)

• In some cases dependencies are there in theenvironment (like global python or node packages)• In some cases dependencies are for testing/dev(example: storybook in Javascript)• In some cases dependencies are removed from theficode but not from the package manager manifests

Your manifest can lie• “Just pip install a dep”
• Baselines won’t function without the right version• You’ll never see it in a manifest or lock le• SCA / dep tree tools (usually) won’t see itfi

OpenAI’s Baselines libraryModels suggest thispattern oftenThe Phantom Dependency Menace• Dependencies that are either “provided”
by the system are assumed to be
downloaded manually• Scripts, containers, and so on• Often depend on the target platform• Dependencies that are required forbuilding an application that are notsupposed to be used at runtime but are
actually used

• Very common in NPM: see storybook forexampleLots of AI Code

Security Challenges• False sense of security — tools can’t see what’s notin a manifest, so you miss risks that might berelevant• Inaccurate compliance data — your SBOMs aren’treporting everything in use. Auditors are unhappy ifthey catch you• Dev / prod di erences — can’t rely on the version Iff

see in dev pipelines being the same thing that’s inproduction

Why are tools blind?
Many tools trust the manifest or lock les, and don’t
account for the ways those can lie
1. Phantom Dependencies (false negatives)• Brought by the system, runtime or other scripts
2. Mis-used dependencies (false negatives)• Dependencies brought as “test/dev” used in runtime
3. Direct use of transitives (unreliable xing)• Dependencies brought in as transitives and useddirectly without knowledge

4. Unused dependencies (false positives and noise)• Dependencies brought in the manifest but not usedfifiby the codeProgram Analysis
FTW
What matters is which packages the code actuallyuses
1. Source of truth is actually the source codea. Analyze the codeb. Create an Abstract Syntax Treec. Analyze types and call ows
d. Create a call graph
2. Correlate the dependencies used by the code withthe dependencies fetch by the package manager oravailable in the le system

flfifi

3. Create a uni ed view

Example: Python
Use the source
1. Import dependency ▶ Call graph ▶ “Is it used?”a. Repeat for all it’s dependencies (transitive)
2. Compare dependency graph with versions installedon system and de ned in manifest
3. Correlate and unify results

fia. Makes accurate SBOM and VEX possible

The Shameless Pitch• Accurately identify all dependencies in use, even ifthey’re “phantom”• Provide clear mapping and pathway data
• What uses it?
• Directly vs. Transitively?
• Find out when transitive deps are being directlyused• Avoid the noise to devs by knowing whether a risk
is actually along a call path

Greetz and Thanks
Dimitri Stiladis, Henrik Plateoriginal research, program analysis design
Jamie Scottmodels, reviews
Antonella Commiato and Pia Rodil
Food and logistics support for this meeting

HiveWatch
Hosting and facility
OWASP Membership and OWASP LA
None of this is possible without the commitment of members and leaders at OWASP
Many, many more…