DOI: 10.3886/icpsr35258
title: Blind Collaborative Justice Survey, 2014 [UNITED STATES]
title: Archival Version

description: These data are part of NACJD's Fast Track Release and are distributed as they there received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except of the removal of direct identifiers. Users should refer to the accompany readme file for a brief description of the files available with this collections and consult the investigator(s) if further information is needed.


This study conducted an experimental investigation of two potential contributors to invalid testimony within adversarial litigation involving forensic evidence. First, the experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and secondly the lack of input from the relevant scientific community. The study used an experimental survey design with a realistic criminal case to examine the effects of blinding experts to their party representation and consensus feedback from a panel of experts to quantify and reduce testimonial bias.
description: The purpose of this study was to evaluate the ability of expert blinding and consensus feedback to improve the validity of expert testimony, specifically in the context of forensic science.
description: The study conducted an experiment that consisted of developing a hypothetical criminal case with forensic evidence, carrying out an exercise with a panel of relevant experts to generate a consensus interpretation of the case evidence. The consensus exercise used a questionnaire consisting of the case description and the forensic question. Two rounds of the consensus exercise was needed to reach the unanimous solution, and this solution comprised the feedback for the justice survey.


The justice survey was conducted with a new sample of scientists using the consensus interpretation. The survey was designed and staged on SurveyMonkey, a web-based commercial tool for survey design and administration. Survey participants were randomly assigned to one of three types of party representation (prosecution, defense, or blinded). Approximately half the participants in each condition received consensus feedback in advance of providing a response, and the other half did not. In addition, the group without advanced feedback received an opportunity to change their baseline response following receipt of consensus feedback (a pre/post within-subjects measurement). After the completion of the survey, respondents were invited to a independently hosted website, SelectSurvey.net, to request compensation. This procedure allowed the partition of identifying information from survey responses.
description: Consensus exercise data (JusticeConsensusExerciseDataCollectionInstrument_20140716163600.pdf, n=12) consists of expert's answers to a forensic question.


Justice survey data consists of two Excel files.


N700_RawData_May1_2014 - Archive3.xlsx Excel file contains three worksheets.



Values worksheet (n=700, 35 variables) and Labels worksheet (n=700, 33 variables) consists of nearly the same variables with the Labels worksheet's values coded using the coding scheme. These worksheets contain variables on respondent's answer to a forensic question and their confidence in their answers, if a consensus response influenced their final answer, opinion on the survey, how much they would ask for payment for consulting for the prosecution and defense, if their answer would favor the party that hired them, how many times they served as an expert consultant for prosecution and defense, age, gender, race, highest degree earned and field, and other comments on the survey. Only in the Values worksheet are variables on survey exclusion and reason.
; 

Summary worksheet contains summary on number of respondents per between-subjects survey condition and a diagram of the conditions.
; 


N580_ProcessedData Oct2014 - Archive.xlsx Excel file (n= 580, 57 variables) contains one worksheet Merged Data. This worksheet contains the same variables as the Labels worksheet in the N700_RawData_May1_2014 - Archive3.xlsx Excel file with the addition of variables on type of survey taken; response to question and confidence level; if respondent has a doctoral degree, has experience, has a forensic degree; and statistics on respondent's answer.
description: None
description: Presence of Common Scales: 
					Likert-type scale was used.
description: Response Rates: 
					82.9% of people who accessed the survey landing page submitted response data and were thus defined as participants.
description: Datasets: 
					
						DS1: Dataset
description: computer-assisted personal interview (CAPI), web-based survey
					These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.
description: People with an advanced degree in a relevant scientific field where training would normally include exposure to Bayes theorem or equivalent experience.
				   		Smallest Geographic Unit: None
description: Consensus exercise: Twelve RAND employees who held doctorates in behavior decision theory, biostatistics, clinical psychology, criminal justice, economics, experimental forensic psychology, industrial and organizational psychology, social psychology, psychology, and statistics.


Justice survey: Fourteen professional scientific societies were targeted. Eligible societies were those with a focus on social/behavioral sciences or statistics and with a primarily United States membership. Nine societies agreed to distribute the survey invitation to their membership list via e-mail or newsletter. 685 accessed the survey and 580 submitted completed surveys.