U.S. flag
      An official website of the United States government, Department of Justice.
        Here's how you know
      Here's how you know**Official websites use .gov**
            
            A **.gov** website belongs to an official government organization in the United States.
      **Secure .gov websites use HTTPS**
            
            A **lock** (
            A locked padlock
            ) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. Menu

      
        close

            
            
                                              About NIJ
        
                    
  
          About NIJ
      
    
          The NIJ Director
          How NIJ Is Organized
          Mission and Guiding Principles
          Research and Development Process
          Strategic Framework
          Assessments of NIJ
          Annual Reports & Reports to Congress
          Staff Directory
          Funding & Awards
        
                    
  
          Funding & Awards
      
    
          Current Funding
          Forthcoming Funding
          Expired Funding
          Awards: Listing of Funded Projects
          Guidance for Applicants and Awardees
          Fellowship & Student Programs
          Travel Scholarships
          NIJ Challenge Program
          Library & Multimedia
        
                    
  
          Library & Multimedia
      
    
          NIJ Journal
          Podcasts
          Publications Listing
          Multimedia Listing
          Rated Practices on CrimeSolutions
          Rated Programs on CrimeSolutions
          Software & Databases
          Datasets
          Patents
          Topics
        
                    
  
          Topics
      
    
          Article Listing
          Corrections
          Courts
          Crime Prevention
          Crimes
          Drugs and Crime
          Equipment and Technology
          Forensic Sciences
          Youth Justice
          Justice System Reform
          Law Enforcement
          Tribal Crime and Justice
          Victims of Crime
          Events
        
                    
  
          Events
      
    
          NIJ 2024 Research Conference
          Scheduled Events
          On-Demand Events
          Research Meetings & Workshops
          NIJ-Hosted Online Training

                            
                                About
          Contact Us
          Subscribe
          Topics A-Z
                  Share
FacebookXLinkedInRedditPinterestEmail Home
                            Funding & Awards
 Panacea or Poison: Can Propensity Score Modeling (PSM) Methods Replicate the Results from Randomized Control Trials (RCTs)?
Award Information
Awardee
              Portland State University
Award #
              2016-R2-CX-0030
Funding Category
              Competitive
Location
                    Portland, OR
Congressional District
              1
Status
              Closed
Funding First Awarded
              2016
Total funding (to date)
              $39,972
Original Solicitation
              Data Resources Program 2016: Funding for Analysis of Existing Data
Description of original award (Fiscal Year 2016, $39,972)
Purpose: Criminal justice agencies rely on research to inform decisions related to policy and practice. Regardless of institutional focus (i.e., criminal victimization, law enforcement, corrections, and federal justice), identifying the practices that will best reduce crime and victimization requires ample research with strong methodology. Understood as the "gold standard" in research design, the randomized control trial (RCT) has been shown to provide reliable and valid findings. However, RCTs are often not feasible in many criminal justice settings because of ethical and practical considerations. As a substitute for RCTs, researchers have increasingly had to rely on other quasi-experimental designs and statistical techniques. One of the more popular techniques is propensity score modeling (PSM), which is designed to simulate the effects of an RCT. With the growing popularity and technological ease of using PSM, this research project seeks to address the critical question: Can PSM methods replicate the results from RCTs?

Research Design and Methods: Using a selection of ten RCT databases available in the National Archive of Criminal Justice Data (NACJD), this study will employ five commonly used PSM techniques to these datasets including (1) one-to-one matching, (2) one-to-many matching, (3) optimal matching, (4) inverse probability of treatment weighting (IPTW), and (5) stratification on the propensity score.

Analysis: For each of the datasets included in this study, a common metric effect size (ES) (i.e., Cohen's d) will be calculated along with a 95% confidence interval (Cl) for the five PSM techniques. The ES results will then be compared to those from the RCT. Following these individual comparisons, this project will also meta-analyze the compared differences in findings. The focus of our analyses will be on assessing the extent to which the Cls overlap with one another. This strategy will be instructive in determining whether there are significant differences found in the ES estimates of the various methodologies used.

Products: The findings from this study will provide evidence to either support or revisit the reliance on PSM in criminal justice program and policy evaluations. This study will also help criminal justice agencies make more informed decisions about which types of research methodologies to invest their limited resources. As a result, multiple peer-reviewed publications are expected from each test of PSM and meta-analysis. Similarly, reports will be disseminated to NIJ and presented to state and federal agencies to determine how best to interpret and apply PSM related research findings. ca/ncf



  
Grant-Funded Publications
        
                      CAN PSM STRATEGIES REPLICATE RCT
                  Panacea or Poison: Can Propensity Score Modeling (PSM) Methods Replicate the Results from Randomized Control Trials (RCTs)?
                  Panacea or poison: Assessing how well basic propensity score modeling can replicate results from randomized controlled trials in criminal justice research
                  Revisiting Washington State's Swift-and-Certain Initiative
                  Gender and community supervision: Examining differences in violations, sanctions, and recidivism outcomes
                  A Supervision Policy with Scope: Revisiting Washington State's Swift-and-Certain Initiative
                  Panacea or Poison: Can Propensity Score Modeling (PSM) Methods Replicate the Results from Randomized Control Trials (RCTs)?




  
Grant-Funded Datasets
        
                      https://doi.org/10.3886/ICPSR37291.v1

  
Date Created: September 14, 2016
    
          Side navigation
  

        

  
  
              Current Funding
    
      
            
            
            
      Forthcoming Funding
    
      
            
            
            
      Expired Funding
    
      
            
            
            
      Awards: Listing of Funded Projects
    
      
            
            
            
      Guidance for Applicants and Awardees
    
      
            
            
            
      Fellowship & Student Programs
    
      
            
            
            
      Travel Scholarships
    
      
            
            
            
      NIJ Challenge ProgramSimilar Awards
        
                      NIJ Innovations in Measuring Community Perceptions Challenge Winner: MCHawks
                  Strengthening Scientific Foundations for Advancing Best Practices in the Collection, Storage, Analysis, and Interpretation of Organic and Inorganic Gunshot Residues
                  Nanoparticle-Decorated Micropillars As Integrated Raman Spectroscopy and Mass Spectrometry Substrates for Quantitative and High-Throughput Toxicology Drug Analysis
 Top

      National Institute of Justice

        
        
                        About NIJ
                    About NIJ
      
    
                  How NIJ Is Organized
      
    
                  NIJ Director
      
    
                  Research Disclaimer
      
    
                  Funding & Awards
                    Funding & Awards Home
      
    
                  Awards Listing
      
    
                  Current Funding
      
    
                  Guidance for Applicants and Awardees
      
    
                  Resources
                    Events
      
    
                  Podcasts
      
    
                  Publications
      
    
                  Topics
      
    
                  Multimedia
      
    
                  Reuse Policy
      
    
                  Contacts
                    Contact NIJ
      
    
                  Staff Directory
      
    
                  Submit a FOIA Request
      
    
                  File a Civil Rights ComplaintFacebook
          
                      X
          
                      YouTube
  

      810 7th St. NW, Washington, D.C. 20531
      
              Secondary Footer link menu
            Accessibility
              Plain Language
              Privacy Policy
              Legal Policies and Disclaimer
              No FEAR Act
              Freedom of Information Act
              USA.gov
              Justice.gov