Computerized Measurement of Treatment Progress for Violent Psychotic Patients: An Application of Generalizability Theory

Jean-Pierre Guay, Brandeis University
Nathalie Auclair, University of Montreal
Frederic Millaud, University of Montreal
Andre McKibben, University of Montreal

ABSTRACT
The emergence of computerized tools is an important advance in clinical assessment. Several methodological problems generally associated with measurement of treatment progress recently led to the development of a Computerized Checklist of Treatment Progress (CCTP). Used with violent psychotic patients at the Institut Philippe Pinel de Montr‚al for more than 2 years, the CCTP is a method of observation in which members of the clinical staff are asked to monthly rate dynamic factors generally associated with treatment progress of psychotic patients (i.e. anger control, empathy, ask for help, attitudes toward medication use, etc.). Eleven scales were developed in order to evaluate treatment progress, and results are used to propose specific individualized goals. The current paper will present the scales used in the CCTP as well as results of inter-rater reliability analyses. CCTP scores will then be analyzed with the use of Generalizability theory (Cronbach, Gleser, Nanda, & Rajaratnam, 1972). Generalizability theory (G) can be used to assess multiple sources of variance (related to subjects, to raters, to treatment setups, to multiple interactions etc.) in a single multivariate analysis and allow decision makers (in this case practitioners) to investigate the dependability of their ratings. Implications for intervention and issues regarding in vivo evaluation's reliability will be discussed along with the advantages and disadvantages of such computerized systems.

References: Cronbach, L.J., Gleser, G.C., Nanda, H. & Rajaratnam, N. (1972) The dependability of behavioral measurements: Theory of generalizability of scores and profiles. New York, John Wiley.

(Return to Program Resources)

Updated 05/20/2006