Statistical models for the generation and 

interpretation of shoeprint evidence

Charless Fowlkes

Dept. of Computer Science

University of California, Irvine
Statistical Reasoning about Footwear Forensics

Goal: develop practical 

statistical models for 

whether a given shoe is 

likely to have generated a 

particular latent print or 

impression.
Statistical Reasoning about Footwear Forensics

Did shoe S leave mark M?

– Is shoe S capable of generating evidence M?

– Given the circumstances C, could some other 

shoe S’ have left mark M?

– Were S and/or S’ present at the crime scene?

Uncertainties:

– Process by which a shoe leaves a mark is 

complex... not every mark left by a given shoe is 

identical.

– We typically don’t have knowledge of all other 

possible shoes S’ that might have produced the 

mark.

Shoe (S)

Impression 

/Marking (M)
Shoe (S)

Impression 

/Marking (M)

Did shoe S leave mark M?

– Is shoe S capable of generating evidence M?

– Given the circumstances C, could some other 

shoe S’ have left mark M?

– Were S and/or S’ present at the crime scene?

How can we evaluate these competing 

hypotheses in the presence of uncertainty?

LR =

P(M | S,C)P(S |C)

P(M | S',C)P(S' |C)

P(M | S,C)

P(S |C)

Statistical Reasoning about Footwear Forensics
Shoe (S)

Impression 

/Marking (M)

Did shoe S leave mark M?

– Is shoe S capable of generating evidence M?

– Given the circumstances C, could some other 

shoe S’ have left mark M?

– Were S and/or S’ present at the crime scene?

How can we evaluate these competing 

hypotheses in the presence of uncertainty?

P(M | S,C)

P(S |C)

LR =

P(M | S,C)P(S |C)

P(M | S',C)P(S' |C)

S'1S

å

Statistical Reasoning about Footwear Forensics
Outline

• Modeling the generation of 

marks from shoes

• Modeling the distribution of 

shoes

• Application: evaluating and 

visualizing reliability of partial 

matching for class determination

Shoe (S)

Impression 

/Marking (M)

P(M | S,C)

P(S |C)
Identifying Characteristics

• Class characteristics:

brand, make, size

• Manufacturing characteristics:

variation across multiple molds, air 

bubbles, assembly, ...

• Acquired characteristics:

wear patterns, cuts and scratches
Statistical Reasoning about Characteristics

Did shoe S leave mark M?

1. Measure features of observed evidence F1(M)

2. Measure features of a particular shoe F2(S)

3. Evaluate if features F2(S) are consistent with 

features of the evidence F1(M)?

Shoe (S)

Impression 

/Marking (M)

P(M | S,C)

P(S |C)

P(M | S,C) » P(F1(M)| F2(S),C)
Measuring Statistics of Characteristics

Challenge: Formalize the 

measurement of identifying 

characteristics so that statistics can 

be computed on large quantities of 

real world data.

Features should correspond closely 

to forensic investigative practice.

F1(M) F2(S)

Class characteristics

Manufacturing characteristics

Acquired characteristics
Motivations for 3D acquisition

• To compute F2(S) we need a digital 

representation of the shoe

• Characteristics of markings are 

fundamentally tied to 3D tread shape

– Contact surface of outsole with hard 

or soft surfaces

– Distribution of acquired 

characteristics (wear patterns, 

accidentals) 

• Methodology and Practice: 

– Assembling datasets for analyzing 

statistics of tread patterns, evaluating 

reliability of features

– Archival documentation of physical 

evidence

Shoe

Photo

Lab Test

Impression

Crime scene

mark
camera 

projector

2D->3D: Structured-light Scanner

• Stereo triangulation between 

calibrated camera pair

• Structured illumination aids 

automatic stereo-correspondence
• Resolution limited (missing fine-scale acquired characteristics) 

• Doesn’t capture shoe shape when it is in contact with surface
GelSight Sensor

Object is pressed against a 

conforming elastomeric gel and 

imaged from below.

Paint on gel surface provides a 

uniform controlled reflectance.

Allows for non-destructive, highresolution recovery of surface shape, 

even for transparent, soft and 

reflective objects

Elastomer

Opaque

Paint

RGB LED camera

Glass

“Retrographic Sensing for the Measurement of Surface 

Texture and Shape”, Micah K. Johnson and Edward H. Adelson. 

CVPR 2009

Outsole
Operating Principle: Shape from Shading

• The intensity of light reflected 

from a surface depends on the 

orientation of the surface relative 

to the light.

– Use multiple colored light sources to 

multiplex several intensity 

measurements into a single color 

image.

– Optimize paint reflectance and 

camera geometry to maximize 

accuracy

• Shape-from-shading techniques 

applicable to a wider variety of 

situations (e.g. images of shoes, 

crime scene impressions)

N
Retrographic Shoe Scanner Prototype

• Sensor surface scaled up to 

cover whole shoe tread from 

commodity components.

• Recapitulates impression 

process.

– Modulate gel stiffness to 

simulate interaction with 

different surfaces

– In principle high-speed 

cameras should allow 

dynamic analysis.. just walk 

across!
3D-> 2D: A simple generative model for marks

Scanned Shoe Model Synthesized Mark

Rigid body simulation to determine 

contact surface.

Assume small elastic deformation 

compressing tread ridges.
From contact surface 

to marks and impressions

Statistics of marks on hard surfaces at 

crime scene depend on:

- modality (paint, dirt, dust, oil, blood) 

- circumstances (not all contact points 

may be “inked”)

Dynamic complexity:

- rolling and slippage of tread against 

rigid surfaces

- interaction of shoe and gait with nonrigid surfaces
Generative model doesn’t need to be 

photorealistic to provide useful feature 

statistics (overall tread pattern, 

location and shape of acquired 

features, wear patterns, etc). 

Going forward... 

We plan to assemble a pilot dataset of 

scanned 3D shoe treads along with 

multiple test impressions

P(F1(M)| F2(S),C)

From contact surface 

to marks and impressions
Outline

• Modeling the generation of 

marks from shoes

• Modeling the distribution of 

shoes

• Application: evaluating and 

visualizing reliability of partial 

matching for class determination

Shoe (S)

Impression 

/Marking (M)

P(M | S,C)

P(S |C)
Prior distribution of shoes

Need to understand what 

alternative shoes S’ might have 

left a mark at a crime scene.

Challenges:

– Unlike, e.g. fingerprints, 

we don’t have large 

databases of candidates

– What statistics are 

relevant to the 

circumstances? 

P(S |C)

[Bodziak, Footwear Impression Evidence, 2000]
Mining the Internet for Tread Patterns

Shoe outsole image dataset (UCI-SHOD)

• Images collected from zappos and 

onlineshoes

– 30,374 shoes 

– 74,016 images

• Shoes appearing on both sites

– 3,549 shoes 

– 20,449 images
UCI SHOD – Inter-class variability
UCI-SHOD -- Intra-class Variations
Limitations

Provides a proxy for understanding 

the diversity of tread features 

related to class characteristics

• Tread images ≠ Shoes

• Statistics do not reflect practical 

investigative circumstances

– Need some additional data

• Relatively few examples of most 

shoe designs

#images per shoe type

 
Outline

• Modeling the generation of marks 

from shoes

• Modeling the distribution of shoes

• Application: evaluating and 

visualizing reliability of partial 

matching for class determination

Shoe (S)

Impression 

/Marking (M)

P(M | S,C)

P(S |C)
Impressions and prints are typically incomplete
Empirical analysis of reliability of 

partial prints in class identification 

Q: How much tread pattern do you 

need to see to reliably determine 

shoe class?

A: Depends on amount of context 

and empirical diversity of tread 

patterns “in the wild”.
Identification from local patches

1. Some patches are more distinctive than others

2. Features in different locations do not contribute independently 

to overall match probability 
Features for matching partial prints

Implement automated image feature 

extractors (normalized correlation, 

edge matching, histograms of oriented 

gradients, deep neural nets)

Evaluate based on class retrieval 

accuracy:

- UCI-SHOD tread images

- Shoe test impressions collected by 

Weisner, et al.

What features should we compute 

from a marking to measure class 

characteristics? F1(M)

Recall

Precision

Normalized Cor.

Squared Difference

Chamfer distance

HOG
Retrieval of matching patches

Query 

patches

Retrieval results (lineup)
dist(patch, reference) 

% reference shoes with

 dist(patch,ref) < t

Larger patches are more distinctive
Visualizing local distinctiveness

distance within same class / distance to other classes
Conclusion

“If you cannot measure it, you cannot improve it”

 --Lord Kelvin

Shoe (S)

Impression 

/Marking (M)

P(M | S,C)

P(S |C)

• Modeling the generation of marks 

from shoes

• Modeling the distribution of shoes

• Application: evaluating and 

visualizing reliability of partial 

matching for class determination
Bailey Kong (UC Irvine) James Supančič (UC Irvine) Deva Ramanan (CMU) 

Sarena Wiesner (Israeli Police)

Yoram Yekutieli (Hebrew University)

Yaron Shor (Israeli Police) 

Acknowledgements