Sarah Bloch-Elkouby’s Poster

Concordance Between Measures of Outcome and Assessment of Possible Deterioration

by Sarah Bloch-Elkouby¹, J. Christopher Muran¹, Bernard Gorman¹, Jeremy Safran¹, & Arnold Winston¹

1 Adelphi University

Objective:

• Performing a multi-dimensional outcome assessment and exploring the relationship between different outcomes measures.
• Exploring the relationship between patients rated vs. therapists rated measures.
• Examining the relationship between different statistical methodologies to assess outcome.
• Exploring different ways of assessing deterioration.

Method:

Table 1: Patient Characteristics

Table 2: Therapist Characteristics

Outcome Assessments:

Results:

Table 4: Termination against drop-out (N=250)

Table 5: Reliable Change Index RCI (Jacobson & Truax, 1991) (N=72)

$$ \text{RCI} = \frac{X_2 - X_1}{SDiff} \hspace{2cm} SDiff = \sqrt{2(SE)^2} $$

Relationship Between RCI for IIP Patients & Therapists

In order to determine the relationship between the RCI calculated for the patients’ IIP and the therapists’ IIP t-tests and correlations were performed:

Clinical Significance (Jacobson & Truax, 1991)

We compared two of the three formulas determined by Jacobson and Truax to calculate the cut-off between functional and dysfunctional populations. Patients were considered recovered if they scored below the cut-off at intake and above the cut-off at termination. They were considered deteriorated if they scored above the cut at intake and below the cutoff at termination.

$$ \text{Formula a: } a = M_1 \pm 2S_1 \qquad \text{Formula c: } c = \frac{SD_1 \times M_2 + SD_2 \times M_1}{SD_1 + SD_2} $$

In order to obtain more comparable results between formula and c, we modified formula a: $$ a’ = M_1 \pm 1S_1 $$

Percent Improvement (PI) Method (Blanchard & Schwarz, 1988)

To examine the relationship between all the outcome measures, we chose the percent improvement method with $$ PI = 100 \left( \frac{X_2 - X_1}{X_1} \right) $$

Relationship Between PI for All the Outcome Variables

In order to examine the relationship between the different outcomes, as measured by the PI, a principal component factor analysis followed by a promax oblique rotation was performed. Two factors, accounting for 57.7% of the variance, were retained for further rotation. The first factor had high loadings on of all the patient measures outcome but negligible loadings from the therapist measures. Conversely, the second factor had high loadings from all of the therapists measures. The correlation between the two factors was small (r=0.26).

Conclusion:

1. Variability across measures :
   • Patients rates of change on the different outcome measures were highly correlated with each other.
   • Rates of change as assessed by the therapists rated measures were highly correlated with each other.
2. Variability across raters:
   • IIP: No correlation between therapists’ and patients’ ratings at intake.
   • No correlation between RCI for the IIP rated by patients and therapists.
   • PI method: Patients and therapists were both consistent in their assessments of improvement but their ratings were not strongly correlated.
3. Variability across methods:
   • Jacobson and Truax: Formula a and c for clinical significance do not yield comparable results.
   • Formula a and a’ may provide more comparable results and need further exploration.
4. Different ways of assessing deterioration were proposed: RCI (most commonly used in the literature), clinical significance, and PI.

Future Directions:

1. Examining if process variables are associated with reliable improvement versus reliable deterioration.
2. Examining if therapist personality variables are associated with reliable improvement versus deterioration.
3. Exploring further and replicate the results of the present study with regard to Jacobson’s and Truax’ formula a for clinical significance.

Acknowledgment:

The cases of the present study were extracted from the archival dataset of the Brief Psychotherapy Research Program at the Beth Israel Medical Center, New York.