U.S. flag
      An official website of the United States government, Department of Justice.
        Here's how you know
      Here's how you know**Official websites use .gov**
            
            A **.gov** website belongs to an official government organization in the United States.
      **Secure .gov websites use HTTPS**
            
            A **lock** (
            A locked padlock
            ) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. Menu

      
        close

            
            
                                              About NIJ
        
                    
  
          About NIJ
      
    
          The NIJ Director
          How NIJ Is Organized
          Mission and Guiding Principles
          Research and Development Process
          Strategic Framework
          Assessments of NIJ
          Annual Reports & Reports to Congress
          Staff Directory
          Funding & Awards
        
                    
  
          Funding & Awards
      
    
          Current Funding
          Forthcoming Funding
          Expired Funding
          Awards: Listing of Funded Projects
          Guidance for Applicants and Awardees
          Fellowship & Student Programs
          Travel Scholarships
          NIJ Challenge Program
          Library & Multimedia
        
                    
  
          Library & Multimedia
      
    
          NIJ Journal
          Podcasts
          Publications Listing
          Multimedia Listing
          Rated Practices on CrimeSolutions
          Rated Programs on CrimeSolutions
          Software & Databases
          Datasets
          Patents
          Topics
        
                    
  
          Topics
      
    
          Article Listing
          Corrections
          Courts
          Crime Prevention
          Crimes
          Drugs and Crime
          Equipment and Technology
          Forensic Sciences
          Youth Justice
          Justice System Reform
          Law Enforcement
          Tribal Crime and Justice
          Victims of Crime
          Events
        
                    
  
          Events
      
    
          NIJ 2024 Research Conference
          Scheduled Events
          On-Demand Events
          Research Meetings & Workshops
          NIJ-Hosted Online Training

                            
                                About
          Contact Us
          Subscribe
          Topics A-Z
                  Share
FacebookXLinkedInRedditPinterestEmail Home
 Using Random Forest Risk Prediction in the Philadelphia Probation Department

August 1, 2012
Review the YouTube Terms of Service and the Google Privacy Policy
See reuse policy
Speakers
        Geoff Barnes, Ph.D.;               Jordan Hyatt, Ph.D.
Watch two experts talk about developing a computerized system that successfully predicts — with a high degree of accuracy — which probationers are likely to violently reoffend within two years of returning to the community.

Drs. Barnes and Hyatt teamed up with the Philadelphia Adult Probation & Parole Department in an NIJ-funded project. Here they discuss:

Why was the Probation Department interested in exploring a new risk prediction tool?
	How does the random forest prediction tool work?
	Why did the researchers have to work hand-in-hand with the practitioners?
	Why is random forest modeling such an effective tool?
  
  
          Transcript
**Geoff Barnes**: The Probation Department had a one-size-fits-all supervision strategy in place. Every offender was getting roughly the same amount, coming in approximately once a month, seeing their officer for 20 to 30 minutes, maybe. It just wasn’t a whole lot of actual contact with the offenders. And, clearly, their budget wasn’t going to go up. They weren’t going to get any more officers.

They really wanted to utilize the resources they had available at the time, which weren’t going to increase, but focus those resources on the people that it made the most sense to focus upon. So identifying those that present the biggest risk to community safety and focusing on them, and at the same time looking at people who really present no risk at all, but had been sentenced to community supervision. Something had to be done with them, but maybe let’s not expend so many resources at the lower end of the spectrum, and instead take what would have been wasted on those people and put it on the people that present the highest risk.

**Jordan Hyatt**: So what this prediction model does is it predicts, using information that the probation department largely had available already, the likely conduct of any probationer for the first two years of their term of probation supervision. So there are three outcomes for this particular model. The lowest level of risk suggests, or it says, that the offender won’t commit any new offenses during that two-year forecasting period. The moderate level of supervision says that the offender will commit a crime, but not a serious crime. And the highest risk of supervision includes those offenders who are forecasted to commit a serious offense, which is generally defined as murder, attempted murder, aggravated assault, rape and arson.

And so what the Probation Department has done is, based on these forecasting outcomes, they supervise offenders in units based on those risk classifications, so the highest risk offenders, those most likely to pose a danger to the community, are supervised most intensely, while the offenders who are predicted to commit no new offenses or relatively minor offenses get a decreased level of supervision.

**Hyatt**: And I think that gets an important point, though: the difference between the researcher side of what we’re doing and the practitioner side. All that the model is doing is saying that based on the information available about this individual, this is what they’re likely to do over the next two years. It’s up to the agency, in our case the Probation Department, to decide what to do with that information.

So here they decided to supervise dangerous people more intensely, but that didn’t have to be the case. That was their decision to make and that’s the policy half of this equation, of this partnership.

**Barnes**: I mean probably the most crucial thing, at least in this project, has to have been that we had researchers and practitioners, but it wasn’t that there was one group on one side and one group on the other. It all had to come together, and it all had to be a partnership, and it had to happen in concert at every single step. We couldn’t have built a model without knowing from them, from our partners at Probation, exactly how many people could they possibly deal with being labeled as high-risk without knowing their capacity to supervise people, and exactly what they wanted to do, and how many officers they could devote to that, without knowing that it couldn’t go much above 15 or 18 percent, we could never have built the model in the first place. Without their data, we could never have built the model in the first place.

**Barnes**: One thing that seems to be a very big improvement in random forest modeling, as compared to stuff we maybe did in the past, maybe stuff that other jurisdictions tried in the past and weren’t very pleased with the results, one of the things is that the amount of information that we can use, you don’t necessarily need to go into the modeling process knowing, well, we think this is important and so we absolutely must go get this.

Lots of different things, lots of different values can be used to predict future behavior, even things, which probably this sounds very strange, but even things that don’t predict future behavior very well, can be included in the model. In traditional statistical procedures, what you typically would have to say is, “Well, we can only have a limited number of predictors to forecast future behavior.” In random forest modeling, you don’t have to be so choosy. You can afford to put things in that maybe don’t work well for older offenders, but work very well for younger offenders.

But I think the important thing is that each individual jurisdiction has access to things probably that they haven’t even thought about. They put this information in as a matter of course, as part of their day-to-day routine, and never really realizing how enormously powerful it could be with just a few edits, with just a few manipulations of it to convert it into a set of numbers that could forecast future behavior.

Well, all this forecasting technology can look overwhelming in a lot of ways. I think if you look at our report, you see something like, ah, well, the model makes nine million different decisions.

**Hyatt**: Yes, 8.4 something million decisions.

**Barnes**: Yeah, you know, you look at it and you say, “How could we ever get to that? It looks so complicated.” But I really think that the reality is different. I think that with the exception of maybe the very smallest jurisdictions, the data are available. It’s just a question of making use of stuff that you already have to build a customized model that fits your particular jurisdiction and your particular needs at the particular phase of the criminal justice system that you are interested in.

**Barnes**: We have to be very careful about how we allocate the precious resource we have, and the most precious resource is time. Every public employee, whether they’re a probation officer or a police officer or a corrections officer, they all only have a certain amount of time that they can devote, and attention that they can devote, to their jobs. We have to make sure we allocate those resources in ways that make sense.

But I think the other reason why we really have to focus on prediction is that, chances are, it’s happening anyway.

**Hyatt**: So probation officers or anybody in criminal justice, people are making judgments about the relative risk that an offender or a probationer poses already. What risk assessment, and specifically actuarial risk assessment, lets us do is ensure that we’re making those predictions in the most fair and equitable way possible. So by using a prediction model like the one in Philadelphia based on random forest modeling, we can be sure that we are identifying the most dangerous offenders in the most accurate way possible, and that we’re doing that in a consistent and fair way. And it ensures that we’re both preserving resources, but also the people who are subject to the policy decisions based on those risk assessments are being treated in a fair and consistent way.

**Barnes**: So one of the reasons to really want to bring this forward in the criminal justice system is that in a lot of ways, it makes the system fairer. It not only is more accurate, but at least you know that however you got put, or if you’re coming into probation and you have to come in once a week because you got put on high-risk probation, at least you know that the decision that put you into that situation was made the exact same way for you as it will be for the guy that comes after you and the guy that came before you, and everyone who comes into probation gets assessed on the basis of the same criteria.

You may not like being on high-risk probation, but from a procedural justice standpoint, you at least know that the decision was made the same way for everybody.
    
  
Disclaimer:
              Opinions or points of view expressed in these recordings represent those of the speakers and do not necessarily represent the official position or policies of the U.S. Department of Justice. Any commercial products and manufacturers discussed in these recordings are presented for informational purposes only and do not constitute product approval or endorsement by the U.S. Department of Justice.
  
Date Published: August 1, 2012
    
          Side navigation
  

        

  
  
              NIJ Journal
    
      
            
            
            
      Podcasts
    
      
            
            
            
      Publications Listing
    
      
            
            
            
      Multimedia Listing
    
      
            
            
            
      Rated Practices on CrimeSolutions
    
      
            
            
            
      Rated Programs on CrimeSolutions
    
      
            
            
            
      Software & Databases
    
      
            
            
            
      Datasets
    
      
            
            
            
      PatentsGender-Based Violence and American Indian and Alaska Native Communities
                  The Changing Threat Landscape of Terrorism and Violent Extremism: Implications for Research and Policy
                  Trauma in Schools - Roundtable Discussion, NIJ Virtual Conference on School Safety
 Top

      National Institute of Justice

        
        
                        About NIJ
                    About NIJ
      
    
                  How NIJ Is Organized
      
    
                  NIJ Director
      
    
                  Research Disclaimer
      
    
                  Funding & Awards
                    Funding & Awards Home
      
    
                  Awards Listing
      
    
                  Current Funding
      
    
                  Guidance for Applicants and Awardees
      
    
                  Resources
                    Events
      
    
                  Podcasts
      
    
                  Publications
      
    
                  Topics
      
    
                  Multimedia
      
    
                  Reuse Policy
      
    
                  Contacts
                    Contact NIJ
      
    
                  Staff Directory
      
    
                  Submit a FOIA Request
      
    
                  File a Civil Rights ComplaintFacebook
          
                      X
          
                      YouTube
  

      810 7th St. NW, Washington, D.C. 20531
      
              Secondary Footer link menu
            Accessibility
              Plain Language
              Privacy Policy
              Legal Policies and Disclaimer
              No FEAR Act
              Freedom of Information Act
              USA.gov
              Justice.gov