Exposing Photo Manipulation with
Geometric Inconsistencies
James F. O’Brien
U.C. Berkeley
Collaborators
Hany Farid

Eric Kee

Valentina Conotter

1

Stephen Bailey

image-forensics-PG14.key - October 9, 2014

Communication by Images

2-1

image-forensics-PG14.key - October 9, 2014

Communication by Images

2-2

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Iranian missile test, 2008

3-1

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Iranian missile test, 2008

3-2

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Iranian stealth fighter, 2013

4-1

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Iranian stealth fighter, 2013

4-2

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Economist manipulates image of
Obama, 2010

5-1

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Economist manipulates image of
Obama, 2010

5-2

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Fabricated image of John Kerry and Jane Fonda, 2004

6-1

image-forensics-PG14.key - October 9, 2014

Image Manipulation

Fabricated image of John Kerry and Jane Fonda, 20046-2

image-forensics-PG14.key - October 9, 2014

Video Manipulation

Flying Birdman Hoax, 2012

7-1

image-forensics-PG14.key - October 9, 2014

Video Manipulation

Flying Birdman Hoax, 2012

7-2

image-forensics-PG14.key - October 9, 2014

Historical Image Manipulation•Image manipulation as
old as photography•Primitive techniques
work surprisingly well
Library of Congress archive
photo of Abraham Lincoln
1826

8-1

image-forensics-PG14.key - October 9, 2014

Historical Image Manipulation•Image manipulation as
old as photography•Primitive techniques
work surprisingly well
Library of Congress archive
photo of Abraham Lincoln
1826

8-2

image-forensics-PG14.key - October 9, 2014

Historical Image Manipulation

9-1

image-forensics-PG14.key - October 9, 2014

Historical Image Manipulation

9-2

image-forensics-PG14.key - October 9, 2014

Image Forensics•Detect forgeries•Detect signs of manipulation•Prove image was modified in some way•Cannot prove an image unmodified!

•Suite of detection tools•Individual methods can be countered by informed attacker•Individual tools may not apply in all cases•Each additional method makes forgery harder

10

image-forensics-PG14.key - October 9, 2014

Advantage: Forgers•People:•Good at understanding scene content•Poor at noticing many types of inconsistencies•Simple manipulation methods work well
•New manipulation methods being developed

11

image-forensics-PG14.key - October 9, 2014

Example Inconsistency

Selected as correct: 62.1%

Selected as correct: 50.1%
N = 20; RT = 7.6s

Farid and Bravo 2010

12

image-forensics-PG14.key - October 9, 2014

Things we don’t see

13

image-forensics-PG14.key - October 9, 2014Things we don’t see

14

image-forensics-PG14.key - October 9, 2014

Advantage: Forgers•People:•Good at understanding scene content•Poor at noticing many types of inconsistencies•Simple manipulation methods work well
•New manipulation methods being developed

15

image-forensics-PG14.key - October 9, 2014

Image Forensics•Format Methods•EXIF meta data
•Quantization tables•Coding decisions•Signatures or watermarks•Not tied to scene content•Easy to apply•Easy to fool (informed attacker)•Not robust to common•Pixel Methods•Linear dependance•Bayer pattern artifacts•Chromatic aberration•Compression artifacts

operations

16

image-forensics-PG14.key - October 9, 2014

Image Forensics•Geometric methods•Content inconsistencies•Require human annotation•Computer analysis•Examples:•Shadows•Lighting•Reflections

17

image-forensics-PG14.key - October 9, 2014

Geometric Image Forensics•Not same as Computer Vision•Possibly user involved in loop
•Only looking for inconsistencies only
•Don’t need to fully extract scene content

18

image-forensics-PG14.key - October 9, 2014

19

image-forensics-PG14.key - October 9, 2014

20image-forensics-PG14.key - October 9, 2014

21

image-forensics-PG14.key - October 9, 2014

22

image-forensics-PG14.key - October 9, 2014

23

image-forensics-PG14.key - October 9, 2014

24

image-forensics-PG14.key - October 9, 2014

25

image-forensics-PG14.key - October 9, 2014

26

image-forensics-PG14.key - October 9, 2014

27

image-forensics-PG14.key - October 9, 2014

28

image-forensics-PG14.key - October 9, 2014

29

image-forensics-PG14.key - October 9, 2014

30

image-forensics-PG14.key - October 9, 2014

31image-forensics-PG14.key - October 9, 2014

32

image-forensics-PG14.key - October 9, 2014

33

image-forensics-PG14.key - October 9, 2014

34

image-forensics-PG14.key - October 9, 2014

35

image-forensics-PG14.key - October 9, 2014

Light in front of camera

Light behind camera

36

image-forensics-PG14.key - October 9, 201437

image-forensics-PG14.key - October 9, 2014

38

image-forensics-PG14.key - October 9, 2014

39

image-forensics-PG14.key - October 9, 2014

40

image-forensics-PG14.key - October 9, 2014

Shading Constraints(c)(b)

41

image-forensics-PG14.key - October 9, 2014

Shading Constraints
a1cb2
365d741253674acbdad8

1bc42

62

573

4
image-forensics-PG14.key- October 9, 2014

1265374

Shading Constraints

cbadad8

1bc625734c8abd43

image-forensics-PG14.key - October 9, 2014

Shading Constraints
212
13a4c4b3abc44

image-forensics-PG14.key - October 9, 2014

Motion in Video45-1image-forensics-PG14.key - October 9, 2014

Motion in Video45-2

image-forensics-PG14.key - October 9, 2014

Parabolic Motion in World (Still Camera)
1p ⌧ = p 0 + t ⌧ v0 + ( t ⌧ ) 2 g2p⌧ = c + ↵⌧ (q⌧ c)p⌧⌧ 2 1..n↵⌧Solve for:

q⌧

v0g

↵⌧

c

46

image-forensics-PG14.key - October 9, 2014

Matching observed motionyzx

47

image-forensics-PG14.key - October 9, 2014

http:// www.youtube.com/ watch?v=WbaH52JI3So

48-1

image-forensics-PG14.key - October 9, 2014

http:// www.youtube.com/ watch?v=WbaH52JI3So

48-2

image-forensics-PG14.key - October 9, 2014

49-1

image-forensics-PG14.key - October 9, 2014

49-2

image-forensics-PG14.key - October 9, 2014

yzx50-1

image-forensics-PG14.key - October 9, 2014

yzx50-2

image-forensics-PG14.key - October 9, 2014

Parabolic Motion in World (Moving Camera)
1p ⌧ = p 0 + t ⌧ v0 + ( t ⌧ ) 2 g2p⌧ = c + ↵⌧ (q⌧ c)⌧ 2 1..np⌧↵⌧
Solve for: v0gq⌧

Track camera motionc⌧51

image-forensics-PG14.key - October 9, 2014

52-1

image-forensics-PG14.key - October 9, 2014

52-2

image-forensics-PG14.key - October 9, 2014

yzx

53

image-forensics-PG14.key - October 9, 2014

Basic Mirror Geometry
Linear perspective image

Image View

MirrorObjectReflection of object

54

image-forensics-PG14.key - October 9, 2014

Basic Mirror Geometry

55-1

image-forensics-PG14.key - October 9, 2014

Basic Mirror Geometry

55-2

image-forensics-PG14.key - October 9, 2014

Basic Mirror Geometry
Mirror-Parallel View
Object Point

Reflected PointnMirror
COP

56

image-forensics-PG14.key - October 9, 2014

Basic Mirror Geometry
Mirror-Parallel View
ObjectReflection

Mirror57

image-forensics-PG14.key - October 9, 2014

Basic Mirror Geometry
Mirror-Parallel View
Bundle of parallel lines

ObjectReflection

In original image they must
converge to a common
vanishing point.
!

(Possibly at infinity)
Mirror58

image-forensics-PG14.key - October 9, 2014

Reflection Vanishing Point
Real Photograph59

image-forensics-PG14.key - October 9, 2014

Reflection Vanishing Point
Real Photographv

60

image-forensics-PG14.key - October 9, 2014

Reflection Vanishing Point
Altered Photograph61

image-forensics-PG14.key - October 9, 2014

Reflection Vanishing Point
Altered Photograph62

image-forensics-PG14.key - October 9, 2014

Reflection Vanishing Point
Altered Photograph63

image-forensics-PG14.key - October 9, 2014

Reflection Vanishing Point
Altered Photograph64

image-forensics-PG14.key - October 9, 2014

Examples

65

image-forensics-PG14.key - October 9, 2014

Examples

66-1

image-forensics-PG14.key - October 9, 2014

Examples

66-2

image-forensics-PG14.key - October 9, 2014

Composite photo World News, copyright 2006.

Examples67

image-forensics-PG14.key - October 9, 2014

Composite photo World News, copyright 2006.

Examples68-1

image-forensics-PG14.key - October 9, 2014

Composite photo World News, copyright 2006.

Examples68-2

image-forensics-PG14.key - October 9, 2014

Photo by Alexi Lubomirski, “The Saint and the Sinner,” copyright 2009.

Examples69
image-forensics-PG14.key - October 9, 2014

Photo by Alexi Lubomirski, “The Saint and the Sinner,” copyright 2009.

Examples70-1
image-forensics-PG14.key - October 9, 2014

Photo by Alexi Lubomirski, “The Saint and the Sinner,” copyright 2009.

Examples70-2
image-forensics-PG14.key - October 9, 2014

Center of Projection•COP determined by 3 orthogonal vanishing points

71-1

image-forensics-PG14.key - October 9, 2014

Center of Projection•COP determined by 3 orthogonal vanishing points
v2

v1

v3

71-2

image-forensics-PG14.key - October 9, 2014

Center of Projection•COP determined by 3 orthogonal vanishing points
v1

v2

v1

C
Image Plane

v3

v2

72

image-forensics-PG14.key - October 9, 2014

Center of Projection•COP determined by 3 orthogonal vanishing points
v1

v2

v1

C
Image Plane

v3

v2

73-1

image-forensics-PG14.key - October 9, 2014

vanishing point established by the rest of the scene wo
the exact location of a given
make the reflection appear unreasonably large or placeor both human error as well
figure too far away to be making the hand off. Even whees. We did not use this same
sible to create a passable reflection, these tests still crea
e of a well-defined reflection
potential stumbling block that raises the difficulty of cit to be necessary. However,Center of Projection
undetectable forgery.
ses where images are highlyason to suspect human error
we COP
dealt withthe difficulty
determinedby
3 orthogonal vanishing points
APPENDIX
simply establishing that nov1
onsistent with the reflectionCv
2of thevscene. A cloud-based
A. COMPUTING THE CENTER OF PROJEC
1Image Plane
ns were all objects in a scene
The center of projection, C, for an image where three
to develop similar methods
orthogonal vanishing points, Vi , have been identifiedmv2. Although very infrequent
the following system of quadratic equations:ments often contain smooth,(C V1 ) · (C V2 ) = 0
such as car bodies, cooking(C V2 ) · (C V3 ) = 0
v3(C V3 ) · (C V1 ) = 0-EPM-2012-01.

•ACM Transactions on Graphics, Vol. 31, No. 1, Article 4, Publication date: J73-2

image-forensics-PG14.key - October 9, 2014

vanishing point established by the rest of the scene wo
the exact location of a given
make the reflection appear unreasonably large or placeor both human error as well
figure too far away to be making the hand off. Even whees. We did not use this same
sible to create a passable reflection, these tests still crea
e of a well-defined reflection
potential stumbling block that raises the difficulty of cit to be necessary. However,Center of Projection
undetectable forgery.
ses where images are highlyason to suspect human error
we COP
dealt withthe difficulty
determinedby
3 orthogonal vanishing points
APPENDIX
v1
simply establishing that no
onsistent with the reflection
Cv2of thevscene. A cloud-based
A. COMPUTING THE CENTER OF PROJEC
1Image Plane
ns were all objects in a scene
The center of projection, C, for an image where threeto develop similar methods
orthogonal vanishing points, Vi , have been identifiedmv2. Although very infrequent
the following system of quadratic equations:ments often contain smooth,(C V1 ) · (C V2 ) = 0
such as car bodies, cooking(C V2 ) · (C V3 ) = 0v3(C V3 ) · (C V1 ) = 0-EPM-2012-01.

•ACM Transactions on Graphics, Vol. 31, No. 1, Article 4, Publication date: J74

image-forensics-PG14.key - October 9, 2014

vanishing point established by the rest of the scene wo
the exact location of a given
make the reflection appear unreasonably large or placeor both human error as well
figure too far away to be making the hand off. Even whe
es. We did not use this same
sible to create a passable reflection, these tests still crea
e of a well-defined reflection
potential stumbling block that raises the difficulty of cit to be necessary. However,Center of Projection
undetectable forgery.
ses where images are highlyason to suspect human error
we COP
dealt withthe difficulty
determinedby3 orthogonal vanishing points
APPENDIXv1
simply establishing that noV2
V1
onsistent with the reflection
Cof the scene. A cloud-based
A. COMPUTING THE CENTER OF PROJEC
Image Plane
ns were all objects in a scene
The center of projection, C, for an image where threeto develop similar methods
orthogonal vanishing points, Vi , have been identifiedmv2. Although very infrequent
the following system of quadratic equations:ments often contain smooth,(C V1 ) · (C V2 ) = 0
such as car bodies, cooking(C V2 ) · (C V3 ) = 0(C V3 ) · (C V1 ) = 0-EPM-2012-01.

•CV3ACM Transactions
on Graphics, Vol. 31, No. 1, Article 4, Publication date: J75

image-forensics-PG14.key - October 9, 2014

NDIX

Center of Projection

OMPUTING THE CENTER OF PROJECTION••

nter COPof projection,C, for an image
where
three mutually vanishing points
determinedby
3orthogonal
nal vanishing points, Vi , have been identified must satisfy
owing
system of quadratic
equations: equations
Systemof quadratic(C(C(CV1 ) · (CV2 ) · (CV3 ) · (CV2 ) = 0V3 ) = 0
V1 ) = 0

(10)
(11)
(12)

.

tions on Graphics, Vol. 31, No. 1, Article 4, Publication date: January 2012.

•Easy to solve by change of variables

76

image-forensics-PG14.key - October 9, 2014

Center of Projection•Building and other structures•Reflectors with rectangular frames!

!

•Frames: two orthogonal vanishing points•Reflected features: third vanishing point•Compare COP from separate
elements in the image

77

image-forensics-PG14.key - October 9, 2014

Center of Projection•Computation is unstable•Step 1: intersect [nearly parallel] lines•Step 2: intersect spheres

78-1

image-forensics-PG14.key - October 9, 2014

Center of Projection•Computation is unstable•Step 1: intersect [nearly parallel] lines•Step 2: intersect spheres

78-2

image-forensics-PG14.key - October 9, 2014

Center of Projection•Computation is unstable•Step 1: intersect [nearly parallel] lines•Step 2: intersect spheres

79-1

image-forensics-PG14.key - October 9, 2014

Center of Projection•Computation is unstable•Step 1: intersect [nearly parallel] lines•Step 2: intersect spheres

79-2

image-forensics-PG14.key - October 9, 2014

Center of Projection•Computation is unstable•Step 1: intersect [nearly parallel] lines•Step 2: intersect spheres• “Instability squared”

79-3

image-forensics-PG14.key - October 9, 2014

Center of Projection•Error sources:•Image resolution•User pointing accuracy•Features from diﬀerent perspectives•COP calculation magnifies error•Structure in instability

80-1

image-forensics-PG14.key - October 9, 2014

Center of Projection•Error sources:•Image resolution•User pointing accuracy•Features from diﬀerent perspectives

Specify regions,
not points•COP calculation magnifies error•Structure in instability

80-2

image-forensics-PG14.key - October 9, 2014

Center of Projection•Error sources:•Image resolution•User pointing accuracy•Features from diﬀerent perspectives

Specify regions,not points•COP calculation magnifies error•Structure in instability*This diagram not to scale

80-3

image-forensics-PG14.key - October 9, 2014

Center of Projection
Real Photograph81-1

image-forensics-PG14.key - October 9, 2014

Center of Projection
Real Photograph81-2

image-forensics-PG14.key - October 9, 2014

Center of Projection
Real Photograph81-3

image-forensics-PG14.key - October 9, 2014

Center of Projection
Altered Photograph82-1

image-forensics-PG14.key - October 9, 2014

Center of Projection
Altered Photograph82-2

image-forensics-PG14.key - October 9, 2014

Center of Projection
Altered Photograph82-3

image-forensics-PG14.key - October 9, 2014

Center of Projection

83-1

image-forensics-PG14.key - October 9, 2014

Center of Projection

83-2

image-forensics-PG14.key - October 9, 2014

Center of Projection

Real Photograph

Altered Photograph83-3

image-forensics-PG14.key - October 9, 2014

CoP from Faces

Work in progress

84

image-forensics-PG14.key - October 9, 2014CoP from Faces

Work in progress

85

image-forensics-PG14.key - October 9, 2014

CoP from Faces

Work in progress

86

image-forensics-PG14.key - October 9, 2014

CoP from Faces

Work in progress

87

image-forensics-PG14.key - October 9, 2014

Summary•Geometric Image Forensics•Human annotation•Computer analysis•Part of “analysis toolbox”•Not always applicable•Together make forgery more diﬃcult•Constrain image content

88

image-forensics-PG14.key - October 9, 2014

Relevant Papers•Eric Kee, James F. O'Brien, and Hany Farid. “Exposing Photo Manipulation from Shadows and Shading”. ACM Transactions
on Graphics, too appear. Presented at SIGGRAPH 2014.
http://graphics.berkeley.edu/papers/Kee-EPM-2014-XX!

•Eric Kee, James F. O'Brien, and Hany Farid. “Exposing Photo Manipulation with Inconsistent Shadows”. ACM Transactions
on Graphics, 32(4):28:1–12, September 2013. Presented at SIGGRAPH 2013.
http://graphics.berkeley.edu/papers/Kee-EPM-2013-09!

•Valentina Conotter, James F. O'Brien, and Hany Farid. “Exposing Digital Forgeries in Ballistic Motion”. IEEE Transactions onInformation Forensics and Security, 7(1):283 – 296, February 2012.
http://graphics.berkeley.edu/papers/Conotter-EDF-2012-02!

•James F. O'Brien and Hany Farid. “Exposing Photo Manipulation with Inconsistent Reflections”. ACM Transactions onGraphics, 31(1):4:1–11, January 2012. Presented at SIGGRAPH 2012.
http://graphics.berkeley.edu/papers/Obrien-EPM-2012-01

89

image-forensics-PG14.key - October 9, 2014

Thank You

90

image-forensics-PG14.key - October 9, 2014