U.S. flag
      An official website of the United States government, Department of Justice.
        Here's how you know
      Here's how you know**Official websites use .gov**
            
            A **.gov** website belongs to an official government organization in the United States.
      **Secure .gov websites use HTTPS**
            
            A **lock** (
            A locked padlock
            ) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. Menu

      
        close

            
            
                                              About NIJ
        
                    
  
          About NIJ
      
    
          The NIJ Director
          How NIJ Is Organized
          Mission and Guiding Principles
          Research and Development Process
          Strategic Framework
          Assessments of NIJ
          Annual Reports & Reports to Congress
          Staff Directory
          Funding & Awards
        
                    
  
          Funding & Awards
      
    
          Current Funding
          Forthcoming Funding
          Expired Funding
          Awards: Listing of Funded Projects
          Guidance for Applicants and Awardees
          Fellowship & Student Programs
          Travel Scholarships
          NIJ Challenge Program
          Library & Multimedia
        
                    
  
          Library & Multimedia
      
    
          NIJ Journal
          Podcasts
          Publications Listing
          Multimedia Listing
          Rated Practices on CrimeSolutions
          Rated Programs on CrimeSolutions
          Software & Databases
          Datasets
          Patents
          Topics
        
                    
  
          Topics
      
    
          Article Listing
          Corrections
          Courts
          Crime Prevention
          Crimes
          Drugs and Crime
          Equipment and Technology
          Forensic Sciences
          Youth Justice
          Justice System Reform
          Law Enforcement
          Tribal Crime and Justice
          Victims of Crime
          Events
        
                    
  
          Events
      
    
          NIJ 2024 Research Conference
          Scheduled Events
          On-Demand Events
          Research Meetings & Workshops
          NIJ-Hosted Online Training

                            
                                About
          Contact Us
          Subscribe
          Topics A-Z
                  Share
FacebookXLinkedInRedditPinterestEmail Home
                            Library & Multimedia
 Panacea or Poison: Can Propensity Score Modeling (PSM) Methods Replicate the Results from Randomized Control Trials (RCTs)?
NCJ Number
              308450

Author(s)
          Christopher Campbell;
                        Ryan M. Labrecque
Date Published

2018
Length
              16 pages
Annotation
              This report provides details of a research study which ultimately demonstrates that propensity score modeling can be a reliable and valid approach to estimating causal inference when conducting a randomized control trial experiment is impossible.
Abstract
              In this paper, the authors report on their efforts to resolve the critical question of whether or not propensity score modeling (PSM) methods can replicate the results from randomized control trials (RCTs) of criminal justice evaluations. The authors focused their efforts on seven different PSM techniques: one-to-one matching, with and without a caliper; one-to-many matching, with and without a caliper; inverse probability of the treatment weighting (IPTW); satisfied weighting scheme; and optimal pairs matching. For their research, the authors gathered datasets of 10 publicly available and restricted RCT studies from the National Archive of Criminal Justice Data (NACJD) which met two baseline criteria for randomization, they introduced an artificial selection bias into the treatment groups of those investigations, and then used each PSM technique to remove that selection bias. The authors describe their research methodology and comparison criteria of the RCT and PSM techniques and present their research results. They discuss how their investigation provides support for the use of PSM in criminal justice research and suggests that those seven PSM methods can be an effective way to estimate reliable and valid simulation of RCT experiments. Based on the findings discussed, the authors emphasize two critical points of caution for anyone using PSM in evaluation research: first, to test multiple techniques to gain confidence in the estimated effect sizes; and second, to assess balance using all appropriate methods and disseminate those assessments. The authors note their research limitations, and conclude that their cross-validation meta-analysis demonstrates clear evidence that PSM can be a reliable and valid approach to estimating causal inference in the absence of the ability to conduct an RCT experiment, however PSM does, in some instances, fall short of the RCT.
      
  
          Additional Details
Grant Number(s)
        2016-R2-CX-0030


  
  
        
  Agencies
        NIJ-Sponsored


  
  
        
  Publication Type
        Research (Applied/Empirical)


  
  
        
  Award
        Panacea or Poison: Can Propensity Score Modeling (PSM) Methods Replicate the Results from Randomized Control Trials (RCTs)?
    
  
  
Date Published: January 1, 2018
    
          Downloads
        
                      PDF

Side navigation
  

        

  
  
              NIJ Journal
    
      
            
            
            
      Podcasts
    
      
            
            
            
      Publications Listing
    
      
            
            
            
      Multimedia Listing
    
      
            
            
            
      Rated Practices on CrimeSolutions
    
      
            
            
            
      Rated Programs on CrimeSolutions
    
      
            
            
            
      Software & Databases
    
      
            
            
            
      Datasets
    
      
            
            
            
      PatentsRelated Topics
        
                Equipment and technology
        Laboratory operations
        Standards development
        Research
        Research methods



  
Similar Publications
        
                      ATM: A distributed, collaborative, scalable system for automated machine learning
                  Reliability of the longitudinal experts all data (LEAD) methodology for determining the presence of elver mistreatment
                  The SECI model and darknet markets: Knowledge creation in criminal organizations and communities of practice
 Top

      National Institute of Justice

        
        
                        About NIJ
                    About NIJ
      
    
                  How NIJ Is Organized
      
    
                  NIJ Director
      
    
                  Research Disclaimer
      
    
                  Funding & Awards
                    Funding & Awards Home
      
    
                  Awards Listing
      
    
                  Current Funding
      
    
                  Guidance for Applicants and Awardees
      
    
                  Resources
                    Events
      
    
                  Podcasts
      
    
                  Publications
      
    
                  Topics
      
    
                  Multimedia
      
    
                  Reuse Policy
      
    
                  Contacts
                    Contact NIJ
      
    
                  Staff Directory
      
    
                  Submit a FOIA Request
      
    
                  File a Civil Rights ComplaintFacebook
          
                      X
          
                      YouTube
  

      810 7th St. NW, Washington, D.C. 20531
      
              Secondary Footer link menu
            Accessibility
              Plain Language
              Privacy Policy
              Legal Policies and Disclaimer
              No FEAR Act
              Freedom of Information Act
              USA.gov
              Justice.gov