Adventures in Developing Risk Assessment Instruments: Research Design and the Impact of Public Policy Concerns in Validation Studies

Diane Blyler, University of Maryland

Risk assessment instruments can be useful tools to assign youth to appropriate levels of supervision and to identify and track dynamic risk factors so youth receive appropriate treatment to assist them with making the changes needed to reduce the risk of recidivism. However, instruments need to be both (1) reliable and (2) valid for the population for which they are used-in this case, juveniles at various decision-making points (e.g., intake, pre-placement) at the Maryland Department of Juvenile Justice. Conducting validation studies lead to some practical and interesting questions: How do public policy concerns get incorporated into a validation study? What preliminary information is most useful to the agency? What validation design makes the most sense for the particular agency? This paper starts with a discussion of basic reliability and validity issues and concludes with a discussion of various issues pertaining to conducting validation studies when working with government agencies that have multiple and sometimes competing concerns.

(Return to Program Resources)

Updated 05/20/2006