Spatiotemporal Correlations in Criminal Offense Records
JAMESON L. TOOLE, Massachusetts Institute of Technology
NATHAN EAGLE, The Santa Fe Institute
JOSHUA B. PLOTKIN, The University of Pennsylvania

With the increased availability of rich behavioral datasets, we present a novel application of tools to analyze
this information. Using criminal offense records as an example, we employ cross-correlation measures,
eigenvalue spectrum analysis, and results from random matrix theory to identify spatiotemporal patterns
on multiple scales. With these techniques, we show that most significant correlation exists on the time scaleof weeks and identify clusters of neighborhoods whose crime rates are affected simultaneously by external
forces.
Categories and Subject Descriptors: J.4 [Computer Applications]: Social and Behavioral Sciences—Sociology; G.3 [Mathematics of Computing]: Probability and Statistics—Correlation and regression analysis
General Terms: Economics, Measurement, Human Factors
Additional Key Words and Phrases: Big data, computational social science, criminology, engineering social
systems, computational sustainability
ACM Reference Format:
Toole, J. L., Eagle, N., and Plotkin, J. B. 2011. Spatiotemporal correlations in criminal offense records. ACM
Trans. Intell. Syst. Technol. 2, 4, Article 38 (July 2011), 18 pages.
DOI = 10.1145/1989734.1989742 http://doi.acm.org/10.1145/1989734.1989742

1. INTRODUCTION

There is an enormous amount of behavioral data generated and stored by billions
of individuals across countries and cultures. It has become necessary to use broader
combinations of tools to analyze this immense and rich stream of information. The
goal is to use this data to gain a better understanding of the systems that generate
it. Insight into these systems can help inform the social sciences from economics tosociology as well as provide policy makers with critical answers that may be used tobetter allocate scarce resources or implement beneficial social programs. In this article,
we present a novel application of tools and analytical techniques developed from avariety of disciplines that may be used to identify patterns and signals that capture
fundamental dynamics of a social system. To capture relationships in both space andtime, cross- and auto-correlation measures are combined with autoregressive models
and results from random matrix theory to analyze behavioral data.
The authors thank NSF Summer REU program for partially funding this research.
Authors’ addresses: J. L. Toole, Engineering Systems Division, Massachusetts Institute of Technology,
1 Amherst Street, Building E40-261, Cambridge, MA 02142; email: jl toole@mit.edu; N. Eagle, The Santa
Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501; J. B. Plotkin, Department of Biology and Programin Applied Mathematics and Computational Science, The University of Pennsylvania, 3451 Walnut Street,
Philadelphia, PA 19104.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or commercial advantage and that
copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights forcomponents of this work owned by others than ACM must be honored. Abstracting with credit is permitted.
To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of thiswork in other works requires prior specific permission and/or a fee. Permissions may be requested from
Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212)869-0481, or permissions@acm.org.
c 2011 ACM 2157-6904/2011/07-ART38 $10.00

DOI 10.1145/1989734.1989742 http://doi.acm.org/10.1145/1989734.1989742
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38

38:2

J. L. Toole et al.

The dataset used in the course of this study consists of criminal events within theCity of Philadelphia from the year 1991 through 1999. It contains nearly one-million
individual criminal offense reports detailing the time, place, and police response totheft, robbery, and burglary-related crimes. In addition to these minor offenses, for theyear 1999, the dataset includes almost every crime, including major offenses, reported
in the city from petty theft through homicide. For these reported crimes, we examine
spatial, temporal, and incident information. The goal of our analysis is to explore thespatiotemporal dynamics of criminal events with the hope of identifying patterns intheir aggregation that may be useful in predicting and preventing future criminal
activity. Beyond applications to criminology, however, we feel that these techniques
can be applied to a wide range of systems that exhibit complex correlations in many
dimensions and on multiple scales.
Existing work from the fields of criminology, sociology, psychology, and economics
tends to explore relationships between criminal activity and socioeconomic variables
such as education, community disorder, ethnicity, etc. [Weisburd et al. 2009; Lafree1999]. In general, constraints on the availability of data meant these studies were limited to aggregate statistics for large populations and vast geographic regions. Wilson
and Kelling’s article in the March, 1982 edition of The Atlantic popularized “broken windows” and “social disorganization” theories that, for the first time, explicitly introduced
flow and system dynamics into crime research. These theories proposed that crime wasa consequence of urban decay and lack of community ownership in neighborhoods andthat these processes worked on a local level [Kelling and Wilson 1982]. Neglected areas
not only attract criminals (the neglect is a sign of lack of police presence), but also act
as a feed-forward mechanism by damaging community morale.
Following the introduction of these theories, attempts have been made to study crimeon the neighborhood level and explore crime “hot spots” [Sampson et al. 1997]. Somestudies have even shifted focus from high crime areas to high risk people, tracking
individuals for a period of time and assessing their propensity to commit crime andits relationship to various socioeconomic indicators [Krivo and Peterson 2000]. Thesestudies tend to be small in size and labor intensive, requiring that neighborhoods besurveyed and tracked for long periods of time.
In this research we address a gap. Statistical methods have been used to characterizelarge, aggregate datasets over long periods of time, while sociological studies have been
performed at microscales. There remains a need for a high-resolution quantitative analysis of large crime datasets. Using offense reports generated by a police department,
we explore how crime here and now affects crime there and then, while also focusing
on building a general set of tools to analyze behavioral datasets for spatiotemporalsystems.
2. DATAThe dataset is used in two parts. The first contains nearly one-million theft-related
crimes from the year 1991 through 1999 while the second consists of almost all twohundred thousand reported crimes within the City of Philadelphia during 1999 acrossall types. In total, crimes were reported at 42,000 unique locations across the city
and were time stamped with the hour they occurred. In addition to time and place,a detailed description of the crime (i.e., theft under $200, aggravated assault with ahandgun, etc.) is provided. A type crime report appears in our dataset as the example
from Table I (note this data as been generated randomly for anonymity purposes andis not an actually report). While there are undoubtedly many interesting questions wemight ask, the primary goal of this research is to better understand the spatio-temporal
structure of criminal events from this data.
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:3

Table I. A Typical Offense Report
Attribute
Offense Number
Date and Time
Address
Lattitude
Longitude
Description of Crime
Place of Event
Police District
Outcome

Data (%)
0000010Saturday, September 12, 1996, 1 p.m.
1700 McKean
39.926152−75.17267917
Theft of car contents, valued at more than $200.
automobile
District 11, Sector B.
The reported crime was confirmed. The case wasclosed without investigation or was not solvable.

Fig. 1. All crimes, major and minor, are plotted on an overlay of census tracts in Philadelphia county during
the year 1999. Geographic features of the city such as rivers, parks, bridges, etc., are immediately visible.

The spatial resolution of this data is high enough that a block/neighborhood analysis
of crime is possible. Plotting the geocoded events reveals features of the city such as thestreet-grid, parks, bridges, rivers, etc. (Figure 1). While the time of each report is known
to within the hour, offenses within a geographic area are generally aggregated to daily,
weekly, or monthly counts, ensuring that time series are sufficiently populated. A timeseries displaying citywide theft-related crimes for 1999 reveals features on multiple
scales. Seasonal trends, such as increases in crime during hot summer months as wellas singular events such as holidays are visible (Figure 2). Finally, when applicable,
offense reports are aggregated by type (Figure 3) so relationships between crimes
can be tested. Though data on crime other than theft are only available for 1999, the
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:4

J. L. Toole et al.
Daily Theft Crime Time Series | 1999
800

Crimes

700
600
500

Thanksgiving

400
Christmas
300

0

50

100

150

200

250

300

350

Day
Monthly Theft Crime Time Series | 1991 1999
10000

Crimes

9000

8000

7000

6000

0

12

24

36

48

60

72

84

96

108

Month

Fig. 2. A time-series plot of citywide theft-related crimes at different time scales. The top figure shows daily
theft crimes for the year 1999 where individual events such as holidays are visible. The bottom figure displays
monthly crime counts from 1991 through the end of 1999, revealing seasonal trends such as increases inthefts during the hot summer months.

two-hundred thousand crimes reported that year still represent a very rich and detailed
dataset with which we can examine interactions between different types of crime.
2.1. Conditioning the Data

Behavioral information must be transformed into variables that can be manipulated
numerically. While time and place readily lend themselves to such analysis, the typeof crime being reported must be inferred from its police description. While the firstpart of the dataset contains theft-related crimes only, for richer 1999 data, crimes areaggregated into six broader categories by parsing police descriptions for keywords as
described in Table II. Aggregation ensures that there are sufficient numbers of events
to populate time series, while still making use of nearly 75% of the data available inthat year.
In order to better measure crime densities, a lattice is laid over a map of the cityand crimes are aggregated to the nearest grid point. In general, any mesh size can beused; our analysis is performed with 50–100 grid locations across the city. This spacing roughly corresponds to neighborhood and census track sizes, allowing for possible
comparison to demographic census tract data. For every lattice point, a time series isconstructed for each category when data is available. With nearly ten years of data
available, time scales from hours to years are examined. These time series are normalized to have zero mean and unit variance to make use of various statistical tests
and to capture changes in crime rates rather than the absolute number of incidents.
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:5

Percentage Breakdown of Crime Type
Burglary 6%

Other 12%Drug 6%

False 9%Assault 12%

Traffic 7%

Auto 7%
Terror 3%
Arson < 1%
Harassment 5%
Vandalism 8%

Graffiti < 1%
Disord. Cond. 1%
Robbery 4%

Theft 20%

Fig. 3. A percentage breakdown of different crimes based on incident reports.
Table II. Categorical Groupings of Different Crime Types forthe 1999 Data
CategoryAll
Automobile
Theft
Violent
Vandalism
Drug

Offenses Included
all reported offenses
auto theft, major traffic
burglary, robbery, auto
assault, homicide, gunvandalism, graffiti
possession, sale, DUI

Crimes (%)
211606 (100%)
16005 (7.6%)80290 (38%)
29908 (14%)
17880 (8.5%)
11642 (5.5%)

Overall, the time series produced are mostly stable and stationary, having roughly
constant mean and variance. Those that have obvious serial correlations are removed
from analysis.
3. METHODS

Using the conditioned data, we will develop analytical tools to achieve the following.
(1) Identify both spatial and temporal relationships.
(2) Choose the best scale on which to examine these relationships.
(3) Interpret the source of the correlation structure.
To address these goals, we combine time-series analysis with results from randommatrix theory to quantify the magnitude and significance of correlation in the data.
In the process, we draw upon knowledge of similar problems found in neuroscience(correlating spike-train data) and financial economics (finding signal in noisy timeseries data) [Kamiński et al. 2001; Mayya and Santhanam 2007; Tumminello et al.
2010; Laloux et al. 1999; Plerou et al. 2002].
Though many methods for detecting correlation in space and time exist, the natureof criminal acts and the type of data being used lent itself to the particular tools used inthis article. For example, while we are looking for correlations in space, it is importantthat techniques are able to detect nonlocal correlations, wherein two neighborhoods on
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:6

J. L. Toole et al.
1999 Crime Density | Drug Crime

1999 Crime Density | Theft Crime

100

40

90
35
80

100

90
100
80

30
70

70

80

25

60

60

50

40

20

50

15

40

60

40

30

30

10
20

20

20

5
10

10

10

20

30

40

50

60

70

80

(a) Drug Crime Density

90

100

0
10

20

30

40

50

60

70

80

90

100

0

(b) Theft Crime Density

Fig. 4. The left plot shows the density of drug-related crimes for the year 1999. Hot spots are located mostly
in central Philadelphia whereas in many theft-related crimes (on the left), are mostly located in the southernareas of the city.

opposite sides of the city are similar in crime profiles because of socio-economic reasons
despite being separated geographically. For this reason, we rule out models that cancapture only local diffusion. Multivariate auto-regressive (MVAR) models that regresstime series of a particular location onto other locations and across many lag periods
can detect such a structure, but given the extremely large number of spatial locations
and the length of time series, these models quickly become too large and intractable[Lütkepohl 2007; Wooldridge 2008].
There is an additional desire to speak of specific details of the correlation structure
rather than just the distribution of crimes in space or time. Models concerned withspeciation in heterogeneous landscapes may provide insight into properties of the
distribution of phenomena or events in nonlinear environments (population density,
socio-economic factors, etc.), but they have more difficulty telling stories of particularrealizations of these distributions.
Finally, there is also the problem of noise. Our dataset, while large, is very noisy andany tools and methods used must be able to find and separate signal with confidence.
The methodology that follows has the advantage that there is both an analytic as wellas computational null model. Random matrix theory provides analytical results on
the limiting distributions and statistics of covariance matrices, while artificial crime
data can reliably be simulated using a poisson process in order to achieve a computational null model. With these considerations in mind, the following algorithm andmethodology was combined to satisfy our modeling and statistical requirements.
3.1. Basic Analysis and Statistics

We begin by using basic data analysis techniques and statistics to better understandthe data. In general, crime is not distributed uniformly across the city. Instead, the spatial structure is consistent with descriptions of crime “hot spots.” These hot spots are
small regions within the city that regularly experience above average levels of crime.
Characterizing the nonuniform spatial distribution of crime is important when attempting to find relationships between types of crime. For example, North Philadelphia
has a high-density of drug-related crimes, while Center City Philadelphia is a hotbed
for theft. We wish to use quantitative tools to measure interactions between locations.
In the time domain, we can identify general periodicities within the data throughbasic Fourier analysis. Ignoring space for a moment, we consider the city in its entirety,
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:7

Single Sided Amplitude Spectrum of y(t)15Once every
three hours
Less
than
once
per
year.
10

|Y(f)|

Once perweek
Once per day

Once permonth
Once per year
5

0
10

1

0

10

1

2

10
10
Frequency (cycles/year)

3

10

Fig. 5. The frequency spectrum of the citywide theft crime time series sampled hourly for nine years from1991 through 1999. Many periodicities appear on timescales ranging from just a few hours to years.

creating a time series of citywide crime sampled hourly. From these methods we canquantify distinct seasonal trends. Cycles exist from hourly to yearly scales. These
periodicities visually coincide with time series showing increases of crime during hotsummer months, or decreases in certain types of crime on weekends that produce
weekly trends. Cycles on smaller scales such as days may come from differences innight and day crime rates and hourly frequencies may be due to police procedure. This
analysis, however, is blind to any measure of auto- or cross-correlation that may occur
between locations within the city.
Running basic regressions on these citywide time series a number of interestingobservations are made. Regressing citywide drug offenses on day of the week, forexample, reveals significant correlation. Considering only the day of the week, we areable to account for nearly 60% of the variance in daily drug offenses (Table III). With
Sunday being the omitted group, coefficients on dummy variables corresponding to
Monday–Saturday are interpreted as the change in criminal activity between Sunday
and that particular day of the week. Thus we conclude Sunday’s have the lowest drugrelated crime rates while the middle of the week (Tuesday, Wednesday, and Thursday)show the highest. This is in sharp contrast to violent crimes, which show an increase on
weekends, dropping during weekdays. We also note that violent crimes are 4–5 times
more frequent than drug crimes.
While in absolute terms most crime occurs during weekdays, observation of these
inverse relationships for certain types of crimes reveals the need to carefully choose theamount of aggregation applied to analysis. It remains unclear, however, if these relationships exist because of some fundamental differences in those committing drug
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:8

J. L. Toole et al.
Table III. A Regression of Citywide Drug andViolent Offenses on Day of the Week
Day
SundayMondayTuesday
WednesdayThursdayFridaySaturdayDrugs
16.55∗∗∗
4.61∗∗
22.52∗∗∗
23.99∗∗∗
21.63∗∗∗
15.52∗∗∗
8.08∗∗∗

Violence
76.87∗∗∗−13.20∗∗∗−10.63∗∗∗−10.71∗∗∗−11.70∗∗∗−4.47∗∗
2.57

2
2(Rdrug= .59, Rviol= .30). ∗∗∗ pval < 0.001,∗∗ pval < 0.05, ∗ pval < 0.1.

Table IV. Impact of Environmental Factors on Crime Occurrence
Const.
Temp. (o F)
Precip. (in)

All
Auto
6.23∗∗∗
3.74∗∗∗∗∗∗
0.0014−0.03∗∗∗-

Drug
3.30∗∗∗−0.078∗Theft
5.37∗∗∗−0.03∗∗

Vandal3.76∗∗∗
0.0018∗∗−0.11∗∗∗

Violent
4.23∗∗∗
0.0026∗∗∗−0.053∗∗∗∗∗ pval < 0.001, ∗∗ pval < 0.05, ∗ pval < 0.1, - Insignificant.

offenses versus violent offenses, or of they are some artifact of police strategy ororganization.
We also note that environmental factors have small, but statistically significant
impacts on crime rates. Using daily weather records in 1999 as kept by the National
Oceanic and Atmospheric Administration (NOAA), we are able to regress crime rates
on environmental factors. While these effects aren’t overwhelmingly strong, they arestatistically significant. We find that temperature increases can be associated with anincrease in crime and that precipitation leads to a decrease. Comparing the coefficientsof these effects for different crime types, we find interesting differences.
In order to compare coefficients between crimes that occur with differing frequency,
we regress the log of occurrences on both temperature and precipitation. The coefficient then represents the percentage change in crime rates due to an increase of one
degree Fahrenheit or one inch of precipitation, respectively. We find that drug-related
crimes, which may be driven by psychological or physiological needs, are not affectedby weather, while violent crimes, which may be driven by passion and environment,
respond significantly to increases in temperature or precipitation.
Though these basic statistics provide insight into the types of relationships that exist
within the data, they remind use that complex relationships exist on multiple scales
in both space and time. We continue with more advanced methods, capable of teasingthese relationships out despite noisy data.
3.2. Auto- and Cross-Correlation

In order to perform higher-resolution analysis, we make use of the scalable latticedescribed in previous sections. Having laid this lattice over the city, we create a timeseries for each point. The time series are then normalized. We select two time series,
y1 and y2 , from the conditioned data and look for correlations between them.
The cross-correlation, r1,2 , is a measure of similarity between a pair of time series.
Mathematically, this quantity is defined as thenexpectation of the inner product between
the two time series: r1,2 = E[y1 , y2 ] =t=1 y1 (t)y2 (t). Similarly, it is possible todetermine lagged correlation by shifting one series by a number of lags, m. The lagged
cross-correlation,
r1,2 (m), is given by modification to the previous formula, r1,2 (m) =
n
y
(t
+
m)y
(t).
1
2
t=1
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:9

Auto vs. Theft | Significant Cross-Correlation30

20

Lags (days)

10

0

5

Significant

10

15Node Number

20

25

30

Insignificant

Fig. 6. A matrix displaying significant lagged cross-correlations between automobile crimes and theft crimes.
Because automobile crimes are counted in both categories, we find correlation at zero lag, but almost no othersignificant relationships.

The cross-correlation values are normalized to lie between −1 and 1, where r1,2 = 1 iscorresponds to exact correlation between two time series. A cross-correlation sequence
is defined as the sequence of cross-correlation values over a range of lags. Examining
the cross-correlation sequence for two time-series, we can identify the existence of asignificant relationship as well as quantify its power over a number of lags. Not only
can these measures detect the flow of crime from one area to another, they can also
quantify its speed and direction.
We must also define a plausible test for significant correlations to differentiate between real and random connections. To achieve this, we create a null model for each pairof time series. Each series is randomly permuted, preserving the mean and distributionof counts, but scrambling the order and this any correlations in time. The new crosscorrelation value is then computed. Repeating this process a large number of times,
we construct a distribution of cross-correlation values from which confidence intervals
can be constructed. If the cross-correlation between the original time series deviatesfrom the random distribution at a given confidence level, we consider it significant.
These cross-correlation methods may be used to look for relationships between
crimes. We may ask if an increase in theft-related crimes leads to violent crimes in
the future. For a given node, we create a time series of daily crime rates for each typeof crime for the year 1999. Next, we construct the cross-correlation sequence for thispair of series across a number of lags (in most cases lags up to 30 days were included).
To visualize these correlations we create a matrix where each column represents the
cross-correlation sequence for a given location (Figure 6).
As an example, we have included automobile thefts in both the “Thefts” category andthe “Automobile” category. Unsurprisingly, we see significant correlation between thetwo crime types at exactly zero lag. The lack of significant correlation for other timelags indicates no other significant relationships where theft in one location leads toviolence in that same location at a later time.
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:10J. L. Toole et al.
Zero Lag Correlation Matrix | Monthtly

1

70
0.8
60
0.6
50
0.4
40
0.2
30
0
20
0.2
10
0.4
10

(a) Daily Drug ’99

20

30

40

50

60

70

(b) Monthly Theft ’91-’99

Fig. 7. The first heatmap is the zero lag correlation matrix for drug-related crimes. There appears to be
little spatial correlation and a lack of high correlated locations. The right zero lag correlation matrix showstheft-related crimes sampled monthly from 1991 through 1999. A much stronger correlation signal is seen.

3.3. Correlation Matrices

Having established a measure of correlation and corresponding null mode1 to asses
significance, we seek to couple this analysis with spatial dimensions. We would like todetect correlations not only in time, but also in space.
To do this, we form a K × T matrix, Y, where K is the number of time series
taken from the location and T is the length of each time series. Keeping track ofwhich location each time series corresponds to, we can associate real city locations
with correlations. The delayed correlation matrix for a specific lag m, C(m), is thenconstructed by matrix multiplication C(m) = T1 YYT (m), where T is a regular matrix
Tyi (t)y j (t + m). Note thattransposition. The elements of C are given by Ci j (m) = t=1
m = 0 corresponds to zero lag.
To illustrate this procedure, we test for correlations in drug-related offenses between different neighborhoods across time over the year 1999. Conditioning the data
as described before, time series are constructed for 35 lattice points. These points
are neighborhoods with enough crime to sufficiently populate daily time series. The
zero-lag cross-correlation measuring real-time correlation is displayed in Figure 7(a).
Entry Ci j of this matrix represents the cross-correlation between the time series ofdrug-related crimes from locations i and j. Examining this matrix, no strong patterns or
regions of high correlation are immediately visible. It should be noted that the number
i and j correspond to time-series labels and not actual location coordinate, but thelabeling of neighborhoods is such that locations i and i + 1 are usually close spatially.
Further more, when applying the tests for significance outlined in the previous section,
we find that very few of these correlations can be deemed significant from random
correlation.
The unstructured correlation matrix suggests that daily neighborhood crime rates
may not be highly correlated spatially. This example, though, does not consider any
lagged correlations that may exist between locations. Constructing matrices for lagged
cross-correlation of up to 30 days (1 month), however, reveal similar results. We do
not find any immediate spatial correlation structure or flows across space and time fordaily drug-related crime levels.
The lack of correlation structure only shows an inability to visually identify correlation structure on daily time scales, but says nothing of correlation structure on the
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:11

Max Eigenvalue Across Lags
3
Calculated
Theoretical Max

Mag. of Eigenvalue

2.5

2

1.5

1

0.5
0

5

10

15

20

25

30

Lag (days)(a) Eigenvalue Density(b) λmax vs. t

Fig. 8. (top) Only one eigenvalue, λ1 = 3, can be differentiated from the noise indicated by the solid line.
(bottom) The solid curve is the eigenvalue density of the actual matrix spectra while the dashed curve is thetheoretical prediction from Eq. (1). (b) We plot the maximum eigenvalue of the delayed correlation matrix
for each of 30 lags. For drug-related crimes, we see a very clear periodicity at a frequency of 7 days (1 week).

scale of weeks and months. Performing the same analysis using weekly theft crime
rates from 1991 through 1999, Figure 8(b) reveals much more significant correlation.
When comparing lagged cross-correlation matrices over a number of lags cycles of highcorrelation match yearly seasonal trends seen in Fourier spectrum, but with added
spatial resolution.
Even on short time scales, however, the data is noisy enough that identifying significant patterns may be difficult given a largely random background.
3.4. The Eigenvalue Spectrum and Comparison to Random Matrices

To quantify the varying degrees of correlation in the matrices, we enlist solutions tosimilar problems found in fields such as financial economics (markets, stocks, equities,etc.) and climate forecasting [Tumminello et al. 2010; Laloux et al. 1999]. Work in theseareas suggests that the correlation structure of systems can be characterized by examining the eigenvalue spectra of correlation matrices. Much success has been found testing the significance of these metrics using results from Random Matrix Theory (RMT).
To test for nonrandom structure in our correlation matrices, we consider two relatedgroups of matrices, Gaussian and Wishart. Entries of a Gaussian matrix, G, are drawn
from a standard normal and a Wishart matrix, W, is formed by matrix multiplication ofa Gaussian matrix and its transpose, W = GGT [Edelman 1988]. The key observationis the direct analogy between formulation of the Wishart matrix and cross-correlation
matrix. We use these random matrices as null models to our cross-correlation measures.
Various analytical results for the distribution of eigenvalues of a random Wishart
matrix can be found in Edelman [1988], Sengupta and Mitra [1999], and Utsugi et al.
[2004]. The eigenvalue density, ρ(λ), is defined as the density of eigenvalues below
λ. Given a correlation matrix whose entries are drawn from the standard normal
distribution, the eigenvalue density as K and T go to infinity is given by the MarcenkoPastur Law
Q (λmax − λ)(λ − λmin)
ρ(λ) =,(1)2π
λwhere Q = T /K ≥ 1 and λmax
min = 1 + 1/Q ± 2 1/Q [Laloux et al. 1999].
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:12J. L. Toole et al.

In other words, we can establish significant correlation structure by comparing eigenvalue spectra from the data to those of a random null model. If we find eigenvalues
significantly outside theoretical thresholds, we conclude there is signal buried in thenoise. The eigenvectors associated with these significant eigenvalues are then interpreted as individual factors contributing to the correlation structure of the system. For
example, the largest eigenvalue (and corresponding eigenvector) in the case of financial
data is identified as the “market” factor, having equally weighted components. Mayyaet al. have obtained similar analytical results for lagged cross-correlation matrices[Mayya and Santhanam 2007].
3.5. Daily Drug-Related Crime Rates in 1999

Examining the eigenvalue spectrum of our daily drug data from 1999, we find a weaksignal. The spectra corresponding to the correlation matrix of drug-related crimes isshown in Figure 8(a). While the majority of eigenvalues cannot be distinguished fromnoise, there does exist a large significant eigenvalue.
Next we examine the spectra for a series of lagged correlation matrices. Plotting themagnitude of the largest eigenvalue for each lagged correlation matrix and comparing
this to the largest value expected from random data, we see a strong cyclic signal with aperiod of 7 days (Figure 8). This analysis suggests that significant correlation structure
is present on a weekly cycle.
In addition to the magnitude of eigenvalues, we also examine the distribution of components within each eigenvector. Again borrowing from results in financial economics,
we adopt use of the Inverse Participation Ratio (IPR) to examine the component struc
ture of the significant eigenvectors. The IPR of a vector is given by I P R(νi ) = Kj=1 |ui j |4
[Biely and Thurner 2008]. A large IPR implies that only a few components contributeto the eigenvector, while a small IPR indicates participation of many components. It
is possible to determine clustering structure from such analysis. For example, in financial data, the eigenvector corresponding to the large “market” eigenvalue has a
low IPR, identifying itself as a force that affects all stocks equally. Other eigenvectors,
with larger IPRs, have components that correspond to various sectors of the market[Biely and Thurner 2008]. For crime data, these components correspond to locations
across the city so a cluster of eigenvector components would correspond to a cluster ofneighborhoods.
Examining the IPRs for significant eigenvectors in lagged correlation matrices, ourresults show that the eigenvector corresponding to the largest eigenvalue has a low
IPR and can thus be interpreted as a “market” force. For the remaining significant
eigenvectors, we find that they too have low IPRs, suggesting there is little clusteringor community structure (Figure 9).
Remembering the strong correlation found between day of the week and the numberof reported crimes from basic regressions outlined before, we find that it is possible
recreate this eigenvalue signal by constructing artificial time series using the regression coefficients from Table III and a Poisson random number generator. Beyond thissingle, large market eigenvalue, we cannot distinguish any other significant structurein the spectrum of correlation matrices generated by daily drug-related crime rates.
3.6. Weekly Theft-Related Crime Rates from 1991–1999

Having identified the only a small correlation structure occurring on the scale of days,
we next look at crime on a weekly scale. As shown in Figure 8(b), much more pronounced
correlation exists. For this data, we construct 108 time series corresponding to locations
across the city. Examining the eigenvalue spectrum for the zero-lag correlation matrix
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

× 10

1.5

38:13

IPR vs. lλ i l

–3

–5

× 10
8
6

1

4

IPR

2
0
0

0.5

0

0

0.5

1

1.5

0.05

|λ |

0.1

2

0.15

2.5

0.2

3

3.5

i

Fig. 9. A plot of the IPR of the eigenvectors of the delayed correlation matrix for drug crimes with a 7-day lag.

Eigenvalue Density

Density

1.5

1

0.5

0

0

2

4

6
Eigenvalue Magnitude

8

10

12

Fig. 10. The eigenvalue spectrum of the zero-lag correlation matrix from weekly counts of theft-related
crimes. The solid line represents the RMT prediction while the dashed line is a fit to the actual distribution
using the min and max eigenvalues as fitting parameters.

of weekly theft data from 1991–1999, we find more significant eigenvalues than in thedaily drug data (Figure 10).
Again looking at the IPR values for these eigenvectors, we find that the large market
eigenvector persists, but now there are other significant eigenvectors (both above andbelow the predicted min and max) that tell different stories. Figure 11(a) shows eigenvectors associated with large eigenvalues generally have low IPRs that correspond tofactors that influences more spatial regions, while significant small eigenvalues havethe highest IPRs, suggesting they are generated by a acting on only a few components(locations around the city).
Looking closer at the eigenvectors that contribute to the information contained inour correlation matrices, we examine the components of each, ui = (u1 , u2 , . . . , uk). The
theoretical distribution of components as derived from RMT is a normal distribution
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:14J. L. Toole et al.
108

107

1.4

0.8

1.2
0.6

1
Inverse Participation Ratio

0.8
0.4
0.6

0.06

0.4

0.2

0.2

0.05

0

4

2

0

2

4

0

4

2

0

2

4

2

4

0.04

Density

IRP

101

87

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.03

0.02

0.01

0

0
0

2

4

6
Eigenvalue

8

10

12

4

(a) IPR202Component Value

4

0

4

2

0

(b) Component Density

Fig. 11. (a) IPR values for eigenvalues associated with the zero-lag correlation matrix for weekly thefttime-series. Points outside of the gray area represent significance outside of the bounds predicted by RMT.
There are more significant eigenvalues, and these results suggest larger eigenvalues with have low IPRs,
corresponding to global forces that affect crime across the city, while small eigenvalues are associated withsignals generated from just a few locations. (b) Component density distributions of significant eigenvalues(λ108 , λ107 , λ101 ) confirm our interpretation of a “market” eigenvalue acting on nearly all locations with thesame bias. Selecting an eigenvector from the random part of the distribution (i.e., λ87 shows good agreementwith theoretic predictions).

with zero mean and unit variance [Plerou et al. 2002].
1
2
ρ(u) = √ e−u /2(2)2π
For the correlation matrices of weekly theft time series from ’91–’99 at 108 city locations, Figure 11(b) compares the component distribution to the theoretical distribution.
We find that the eigenvector associated with the largest eigenvalue has nearly all positive components. Coupling this with the low IPR, we see why it is possible to identify
it as a market force. It has a large positive bias across all locations of the city. Othersignificant eigenvectors also have nonuniform distributions. We compare these to an
eigenvector from the random part of the spectrum that shows good agreement withRMT results.
In order to accurately quantify and interpret structure in the remaining significant
eigenvalues, we must first remove the dominating influence of the largest. Because thestrong “market” eigenvector acts as on all locations across the city with the same bias,
we can recreate this global influence by projecting the citywide crime time series onto
the eigenvector u108 .1 This time series can be viewed as an estimate of citywide crime
based on the most prominent factor. Denoting the original normalized time series asy(t), we construct the projection.
y108 (t) =

108


u108
j y j (t)

(3)

j=1

Comparing this time-series with the original weekly citywide crime we find strong
agreement over nearly ten years of weekly data with the correlation coefficient of
Y (t) Y108 (t) = .95.
1 This procedure is outlined by Plerou et al. [2002].

ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:15

Having established a reasonable proxy for the market forces acting on crimes rates at
all locations, we regress the location time series on this global force to and use residuals
that are free from its influence. For the time series associated with each location yi (t),we perform the following regressionyi (t) = αi + βi Y108 (t) + i (t),(4)

where αi and βi are location-specific fit parameters. The residual time series, (t) arethen used to compute the same correlation matrices and spectral analysis as described
previously, but this time with the absence of global trends.
We now take the significant eigenvectors of the residual correlation matrices andexamine their component structure. Large components of specific eigenvectors correspond to locations across the city that are all similarly biased by whatever force isassociated with the vector. When we plot the largest 10% of components for the remaining significant eigenvectors geographically, we find that large components of each
vector are strongly correlated spatially.
Figure 12 shows the spatial distribution of large components for different residual
eigenvectors. The vector associated with the second largest eigenvalue, u107 , acts primarily on neighborhoods in high crime areas near south-central Philadelphia, while
the next largest is made up of three neighborhoods located on different edges of thecity. Taking the random eigenvector, u87 , we see that the locations are distributed more
uniformly in space.
Analysis on this weekly reveals a rich structure spatially. Performing similar procedures using monthly time series reduces the amount of correlation, suggesting thatthe weekly time scale is the correct choice for analysis of neighborhood crime trends.
The problem of scale selection is one that can be dealt with naturally given the algorithms applied in this article. While Fourier analysis, regressions, and cross-correlation
measures all give indications as to the amount of signal in the data, none provided ouranalysis with a satisfactory selection of scale. For example, the largest frequency in the
FFT of citywide theft counts is found at once-per-three-hours. This result most likely
has something to do with police procedures such as shift changes, but selecting thehourly scale for all subsequent analysis would surely return time series too noisy andunpopulated for use.
By using results from random matrix theory as a null model, we can easily measure
how much signal can be distinguished from noise by observing eigenvalues above
predicted maximums. Furthermore, changes in both spatial and temporal scales affectthis spectrum in the same way: increasing or decreasing the number and magnitudeof significant eigenvalues. We can select scales by sweeping over time from hours tomonths, and space from a small lattice to a large one, and track only the number
and magnitude of significant values. Selecting the combination that maximizes these
deviations allows us to extract that most signal from noisy data, aggregating as much,
but not more, than necessary.
We can refine these statements even further by noting that whatever social phenomena is behind these significant eigenvectors (or principle factors), they are independent
in some way. Whatever force is driving crime rates in the locations corresponding
to large components of the second eigenvector is separate enough either in cause or
manifestation from the force behind the third vector. This is a level of interpretation
currently not offered by the majority of tools aimed at explaining causal relationships.
The underlying social dynamics of these factors, however, remain unknown, a point
we hope to address in future work. What is important, however, is that this analysis
suggests that there is good reason to treat crime differently within different neighborhoods and policing strategy may be improved by introducing more local autonomy into
strategy.
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:16

J. L. Toole et al.
107

106

20

20

15

15

10

10

5

5

0

5

10

15

20

0

5

10

Normalized Latitude

87

20

15

15

10

10

5

5

5

10
15
Normalized Longitude

20

15

20

104

20

0

15

20

0

5

10

Fig. 12. Plotting the geographic location of the largest 10% of components for each significant eigenvector
reveals strong spatial correlation. We conclude that these eigenvectors correspond to neighborhoods or setsof neighborhoods and represent the forces that affect crime rates there. Examining random eigenvectors(bottom left), we see that the components are distributed nearly uniformly spatially.

4. SUMMARY AND CONCLUSION

In this article, we have presented a novel application of quantitative tools from fields
such as mathematics, physics, and signal processing that can be used to analyze spatial
and temporal patterns behavioral datasets. Basic analytic and statistical techniquesrevealed periodicities and cycles within the data that were then explored further byhigher-resolution techniques. Due to a low signal-to-noise ratio, we adopted resultsfrom random matrix theory to construct a suitable null model to construct significancetests. These tests revealed definite structure in the eigenvalue spectra of our correlation
matrices.
Given the large portion of daily crime rates that can be explained by regressing data
onto the day of the week, it is possible that these results reflect police procedures suchas scheduling more officers on Mondays than Sundays. Differences in high crime days
across types of crime, however, may suggest different types of crime may represent
unique aggregate behaviors. Another interesting result from our analysis is the lack ofcorrelation between these different crime types. Broken windows and social disorganization theories postulate that an influx of minor offenses such as graffiti and vandalism
might lead to an increase of more serious crimes such as assaults or gun violence. We
find no evidence of this. This is not to say, however, no relationship exists. Due to data
constraints, we have only looked for interaction on time scales of up to 30 days. It may
be that these types of flows happen on the monthly or yearly time scale.
ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

Spatiotemporal Correlations in Criminal Offense Records

38:17

With ten years of theft crime data, we are able to look for correlations on a weekly
scale. Examining the eigenvalue and eigenvectors of correlation matrices, we perform
a sort of principle factor analysis where significance is determined by comparison toresults from random matrix theory. We find that the largest eigenvalue and corresponding eigenvector can be interpreted as a citywide force that acts on all locations
equally.
Removing the influence of this dominating factor, we find a number of significant
eigenvalues and vectors. Examining the component structure of these eigenvectors,
we find that they correspond to neighborhoods or sets of neighborhoods that sharecorrelated crime rates. We draw an analogy to similar methods used in the analysis
of finance markets where the eigenvalue spectrum also contains a “market” influence,and other significant eigenvalues are associated with specific industries and market
sectors. The added value in this type of analysis is the ability to identify correlations
due to different eigenvectors as representing independent forces.
We believe there is valuable policy and prevention insight to be gained from thiswork. We have demonstrated that daily crime rates are of little use when attempting
to find significant correlation structure, while weekly crime rates may be a muchricher choice. Along with the selection of a proper time scale, we have identified sets ofneighborhoods whose crime rate are driven in sync. These connections are not entirely
visible to simple correlation measures and may include neighborhoods at opposite endsof the city. They can also be attributed to different forces suggesting policy that workin certain areas may not be relevant to others. Adoption of police procedures that takethese relationships in account may lead to more effective law enforcement.
Aside from the results gained by using these techniques on crime data, we hope thatthey can and will be put to use in more situations calling for multiscale analysis. Not
only do they prove reliable even given very noisy data, but there are strong theoreticand experimental null hypotheses to verify the significance of results. Already, thesemethods have been shown capable of performing reliable factor analysis of financial
markets and crime rates, two very complex behavioral systems.
In the future, we hope to extend these results to find components of temporal correlation as well as the spatial structure shown here. Incorporating lagged correlation
matrices may shed light on lead-lag relationships in the spread of crime across the city.
ACKNOWLEDGMENTS
We would like to thank the Santa Fe Institute and the wonderful community of people who have helped use
with their thoughts and ideas.

REFERENCES
BIELY, C. AND THURNER, S. 2008. Random matrix ensembles of time-lagged correlation matrices: Derivation of
eigenvalue spectra and analysis of financial time-series. Quant. Finance 8, 7, 705–722.
EDELMAN, A. 1988. Eigenvalues and condition numbers of random matrices. SIAM J. Matrix Anal. Appl. 9, 4,
543–560.
KAMIŃSKI, M., DING, M., TRUCCOLO, W. A., AND BRESSLER, S. L. 2001. Evaluating causal relations in neuralsystems: Granger causality, directed transfer function and statistical assessment of significance. Biol.
Cybernet. 85, 2, 145–157.
KELLING, G. L. AND WILSON, J. Q. 1982. Broken windows. The Atlantic.
KRIVO, L. J. AND PETERSON, R. D. 2000. The structural context of homicide: Accounting for racial differencesin process. Amer. Sociol. Rev. 65, 4, 547–559.
LAFREE, G. 1999. Declining violent crime rates in the 1990s: Predicting crime booms and busts. Annual Rev.
Sociol. 25, 1, 145–168.
LALOUX, L., CIZEAU, P., BOUCHAUD, J. P., AND POTTERS, M. 1999. Noise dressing of financial correlation matrices.
Phys. Rev. Lett. 83, 7, 1467–1470.

ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.

38:18J. L. Toole et al.

LÜTKEPOHL, H. 2007. New Introduction to Multiple Time Series Analysis, 1st Ed. 2006. Springer.
MAYYA, K. B. K. AND SANTHANAM, M. S. 2007. Correlations, delays and financial time series. Econophys. Markets
Bus. Netw., 69–75.
PLEROU, V., GOPIKRISHNAN, P., ROSENOW, B., NUNES AMARAL, L. A., GUHR, T., AND STANLEY, H. E. 2002. Randommatrix approach to cross correlations in financial data. Phys. Rev. E 65, 6, 066126+.
SAMPSON, R. J., RAUDENBUSH, S. W., AND EARLS, F. 1997. Neighborhoods and violent crime: A multilevel studyof collective efficacy. Science 277, 5328, 918–924.
SENGUPTA, A. M. AND MITRA, P. P. 1999. Distributions of singular values for some random matrices. Phys. Rev.
E 60, 3, 3389–3392.
TUMMINELLO, M., LILLO, F., AND MANTEGNA, R. N. 2010. Correlation, hierarchies, and networks in financial
markets. J. Econ. Behav. Organ. 75, 1, 40–58.
UTSUGI, A., INO, K., AND OSHIKAWA, M. 2004. Random matrix theory analysis of cross correlations in financial
markets. Phys. Rev. E 70, 2, 026110+.
WEISBURD, D., BRUINSMA, G. J., AND BERNASCO, W. 2009. Units of analysis in geographic criminology: Historicaldevelopment, critical issues and open questions. In Putting Crime in its Place: Units of Analysis in
Geographic Criminology. D. Weisburd, W. Bernasco, and G. J. Bruinsma, Eds. Springer.
WOOLDRIDGE, J. 2008. Introductory Econometrics: A Modern Approach (with Economic Applications, Data
Sets, Student Solutions Manual Printed Access Card), 4th Ed. South-Western College Publishing
Received July 2010; revised August 2010; accepted August 2010

ACM Transactions on Intelligent Systems and Technology, Vol. 2, No. 4, Article 38, Publication date: July 2011.