Practitioners encourage clients to complete between session tasks to help apply behavioural or cognitive, beneficially into their everyday lives. Homework assignments are completed out of session and often require the client to complete a list of cognitive distortions and record their dysfunctional thoughts. This will allow for clients to highlight their thought patterns, emotions and physiological sensations (Kazantzis & Daniel, 2009). Homework assignments provide the client with the opportunity to attain beliefs and reduce negative thought patterns and to further provide therapists with objective and detailed reports (Beck, Rush, Shaw & Emery, 1979). As a prime component of cognitive- behavioural therapy (CBT; Beck, et al. 1979; Ellis, 1962) it allows for clients to engage in homework activities, gather new information, establish new therapeutic skills and incorporate these improved actions from therapy to situations in which their problems occur (Neimeyer, Kazantzis, Kassler, Baker, & Fletcher, 2008). Indeed, research has established that homework during the course of CBT is positively associated with reduced symptom severity and therapeutic benefits (Beck, et al. 1979; Kazantzis, Deane, & Ronan, 2000; Scheel, Hansen, & Razzhavaikina, 2004). This explanation is most consistent with the conceptual dimensions of CBT, as between session-assignments generally focus on and aim to challenge negative thinking (Addis & Jacobson, in press). Several studies show support of the combination of homework with CBT in comparison to without homework (Bryant, Simons, Thase, 1999; Neimeyer & Feixas, 1990). Research (Kazantzis & Dattilio, in press) found that 62% of clients who received homework in CBT showed improvement, as opposed to 38% who didn’t receive homework, indicating its contribution to CBT. More over, in accordance with the Hullian learning theory (Kazantzis & L’Abate, 2007) the inclusion of homework assignments is suggested to be beneficial for clients as they provide structure and clarity to the commonly unstructured and uncertain process of therapy. Despite, the widespread use of homework assignments and support for the causal relationship between homework and treatment outcome, there is limited data that exists on the specific influences of assignments in psychotherapy.

Previous studies investigating CBT and homework have focussed on the positive association between homework outcome and therapeutic homework adherence or compliance (Addis & Jacobson, in press; Scheel, et al, 2004; Westra, Doizois, & Marcus, 2007). Homework compliance is the extent to which the client implements the advised homework, displaying their level of commitment and involvement in the psychotherapeutic process. Individual homework compliance provides an insight into the extent that the client’s behaviour between sessions aligns with the homework assignment previously discussed. More over, Burns & Spangler, 2000 found that there is a linear relationship between homework and outcome, where clients who complete a high quantity of homework and have a high level of involvement in the assignment, show more improvement. There are a number of factors which have shown to produce client compliance including, problem severity, delivery methods by therapists, client characteristics and the assessment of compliance (Scheel, et al, 2004). Practitioners (Kazantzis, Deane and Ronan, 2004) have found that when using independent observer ratings of therapist competence and their accuracy with homework assignments, therapist views of the homework assignments correlated with client competence. However therapists play an important role in creating the clients experience of the homework task, influencing their level of homework compliance. A study by, Hoelscher, Lichstein, & Rosenthal, 1984, discovered the tendency for therapists to reflect inadequate measurements of compliance. For instance, therapists interpreted and overestimated clients as more compliant if they obtained more successful outcomes and on average overestimated homework compliance on 91% of occasions. Therefore, even though there has been empirical support for the positive relationship between homework and compliance, it is difficult to measure compliance as it is counterproductive. This emphasises the importance of investigating other influential factors.

Past research conducted by Kazantzis, Deane and Ronan, (2004) found that when researching the methods of homework and compliance, client self-report was the most common source of homework compliance data, as it was utilised in 28% among 32 studies. This infrequency of comparing two or more types of data leads to inconsistency of results of among studies (Kazantzis et al. 2004, Abramowitz Franklin, Zoellner & DiBernardo, 2002) as it can not be sure that clients will evaluate themselves reliably as they might be influenced by self-enhancement desires. This stresses the importance of integrating both client and therapist perspectives when studying homework compliance, proposing the Assignment Compliance Rating Scale (ACRS) and the Homework Rating Scale (HRS) as valid assessments. The ACRS is a single item rating scale and was devised to assess the quantity of homework compliance and solely measures the quantity of homework compliance, not the quality of the completed assignment. Where as the HRS separates compliance into two distinct components and addresses both measurements of quality and quantity (Kazantzis, Deane, & Ronan, 2005). It is a self report questionnaire (12 questions) and measures client compliance with homework assignments by focusing on client, clinician, and task characteristics. Although the items have been found (Kazantzis, Bjornholdt, Munro, Dobson, Merrick, Fletcher, Jones, James, 2006) to produce good internal consistency and increased consistency over sequences, the inter-item correlations found most items to show variability over time and be subjected to social-desirability bias, that is; the tendency for clients to respond to their clinicians in a socially desirable manner. For that reason a revised version (HRS-II), was devised to improve on the HRS by: reversing the wording of items which measure clients evaluations of their non to sectioned completion of the homework task to increase reliability. This allows for the improvement of the scaling system by utilising biases to highlight areas of difficulty experienced by the client with their completion of homework tasks. This assessment instrument addresses aspects which concern homework completion rather than compliance, examining the theoretically meaningful determinants of homework completion, beliefs (progress matched with therapy goals), consequences (pleasure, mastery and progress) and engagement (quality, quantity, difficulty and obstacles).

When assessing homework assignments and the beneficial attributes to CBT, dependence is placed on the validity and reliability of assessment instruments. This emphasises why the HRS was revised and refinements were made to the theoretical approaches of measurement to increase reliability and validity of ratings. The altered instrument’s rationale encourages involvement of clients in treatment and combined with compliance of homework assignments contributes to further change. An analysis of Cognitive Therapy Scale (CTS) highlights the necessity for psychometric investigation. The CTS-R, was created to improve on the CTS by: abolishing overlapping between items and ensuring that items more described more clearly. The observer-rated scale (11 items) and additional research has confirmed that the CTS-R has high internal consistency and sufficient inter-rater reliability and high levels of validity have been found suggesting improved ratings of competence (Blackburn et al., 2001; Haddock, 2001; Vallis, Shaw & Dobson, 1986). The minimal supply of studies investigating the psychometric properties of the HRS-II, results in little room for psychometric analysis. However, (Kazantzis, N., Bjornholdt, A., Munro, M., Dobson, K. S., Merrick, P. L., Fletcher, R. B., Jones, D., James, Y. H., 2006) completed pilot studies which reported on the psychometric properties of the HRS-II measuring the co-ordinance between client and therapist ratings and the reliability of independent observer ratings. The evaluation found the HRS-II to provide high reliability and validity. Internal consistency of the scale was found to be high with overall coefficient alpha’s of .87, and.84 for client and therapist ratings respectively. Additionally a high Inter-class correlation coefficient of independent observer measures (.82) specified little variation among rater scores, suggesting for high inter rater reliability. Lastly, there was a high correlation evident amongst the scores of factors accounting for good factorial validity. Indeed, as independent observers in this study were trained therapists it needs to be remembered that commonly independent observers ratings might be subjected to the ‘halo effect’ that is; the tendency for raters to mark the therapist high on all categories if they consider them to be of good quality and the reverse is found to also occur (Young & Beck, 1980). Further factors which might influence observer ratings are, contrast effects, similarity and discretion with first impression accuracy, therefore psychometric properties of scales should be considered when examining future studies to aim to account for biases.

Selective attention, confrontation with interference and the concepts of working memory and inhibition are influential factors of task performance, affecting the way one would respond to relevant information whist attempting to disregard irrelevant information. The Stroop task (Stroop, 1935) measures one’s concept of selective attention and inhibition control. Past research (MacLeod, 1991) used this task and instructed participants to name the print colour of a coloured word. For instance, respond ”blue” when exposed to the word green which is printed in blue. This task entails participants to demonstrate inhibition control and not read the coloured word. This proves to be a hard exercise as reading is commonly an automatic process, resulting in a tendency to read the word rather than name the print colour. Therefore the evaluation of congruent and incongruent session content resembles that of a stroop-like effect.

Existing data supports the notion that non-compliance is a common issue in therapeutic sessions. Common areas of concern between the client and homework compliance include, the difficulty they endured with the task, doubting their own individual capability of completing the task or portraying an opposing view about the assigned homework (Fehm & Mrose, 2008). Non- compliance of homework assignments has been linked to the client not attending to or showing involvement in the homework that was assigned to them in previous sessions. Whereas clients who freely expressed feedback and information about their homework presentation showed high homework compliance, which confirms the notion for following up homework in therapy sessions (Shelton & Levy, 1981). However, compliance and non-compliance is difficult to assess. For example, a client may spend an extensive duration of time attempting a homework task, but fail to complete entirety of the task, therefore compliance may not precisely depict the client’s experience of competing the homework. In accordance with the Theory of Planned Behaviour (Ajzen, 1988) studies have found when a client is in agreement with the rationale, they will be more likely to complete their homework assignments as they are depicted as relevant and directive toward a positive change (Addis & Jacobson, in press). A high importance is placed on the clinician interrelated attitude toward the homework, consisting of congruence, positive and encouraging regard followed by empathy. Additionally, Kazantzis, Deane, Ronan & L’Abate, (2005) found that when using independent observer ratings of therapist competence and their accuracy with homework assignments, therapist views of the homework assignments correlated with client compliance. Furthermore inter-rator reliability in measures of competence is affected, as supervisor ratings were linked to positive CBT outcomes but independent ratings who received no additional information did not. The identification of non-compliance with homework can assist in the disclosure of problematic areas highlighting the client’s motivation, commitment and level of participation in the changing process. These elements can then be followed up and different and more suitable strategies can be implemented.

The administration of additional information may lead to the possibility of confirmation bias among independent observer raters. That is; interpreting data in ways that are partial to ones own existing expectations, beliefs, or personal assumptions (Nickerson, 1998). Ditto & Lopez (1992) found that when participants were showed numerous accounts of new information in sequence, which either supported or conflicted with their tentative decisions they overestimated the value of supportive information and underestimated those which were conflicting. Yet there was found to be a positive relationship between the similarities in interpersonal and credibility levels and changes in attitude. In accordance with Source credibility theory (Hovland, 1953) it has been suggested that individuals are more likely to be persuaded and resulting in an attitudinal change by a resource which is deemed as highly credible rather than mildly credible (Aronson, Turner & Carl-Smith, 1963). This theory connotes that perception of trustworthiness and the perception of expertise of the informative source as the two main elements which lead to bias (Strong, 1968). For instance if an individual perceives the source as knowledgeable and highly credible or believes that they source is unbiased, they are more susceptible to change.

However, the present study will be working with client feedback and will involve the independent variables, four possible combinations of two attributes, homework adherence (task completed vs. not completed) and homework attitudes (task appealing/ helpful vs. task not appealing/ unhelpful), followed by the assessment of a further two independent variables, Congruence and non congruence. Congruence will be defined as adherence plus positive attitudes, and non adherence plus negative attitudes. Non-congruence will be defined as adherence plus negative attitudes, and non-adherence plus positive attitudes.

The aim of the research is to examine the effects of session content and additional information on the reliability of independent observer assessments of therapy process. As between-session ‘homework’ tasks are a standard component of Cognitive Behaviour Therapy, the present study will examine the factors influencing ratings of homework tasks. Specifically, the study will examine the Homework Rating Scale – Revised (HRS-II; Kazantzis, Deane, & Ronan, 2005).

It was hypothesised that if the session content is related to reliability in observer ratings, then ratings of sessions with congruent content will be more reliable than non-congruent content. If access to client and therapist data influences observer ratings, then ratings made in the absence of written feedback from clients and therapists will be more reliable.

Participants

A total of 180 men and women, 18 and 65 years of age will be recruited for the study. All participants will be students or staff of La Trobe University, or members of the community, who volunteer for the experiment. Participants will be proficient in reading, writing, and conversing in English and free of current mental illness. These inclusion criteria will be outlined in advertisements and in the study information sheet.    

There is no prior research upon which to estimate the likely effect size of variations in session content or additional information on the factors influencing observer ratings on the HRS. Therefore, the present study will recruit 180 individuals taking alpha at the convention 0.05 level, assuming a small effect for information, and interaction between information and session content.

Recruitment

Participants will be recruited via advertisements, either posted at La Trobe University’s noticeboards and via Learning Management System, or in flyers distributed via an information booth at La Trobe University’s Agora. Discussion about the study will also be made possible from this booth. La Trobe University students will have the opportunity to participate on the same day as recruitment in addition to previously organised dates. Recruitment from the community will occur via the researcher’s social networks and by word of mouth. In all instances, individuals expressing interest will have the opportunity to view a paper advertisement regarding the study outlining the criteria and benefits of participation.

The information sheet will be made available to those who express an initial interest in the research. The information sheet will outline the study and fully inform potential participants about the nature of the study. The researchers’ email and phone number will be available on this sheet, so that those interested may have the opportunity to consider the study, and decide whether to contact the researchers about taking part on the data collection dates.

Materials

            The Homework Rating Scale- Revised (HRS II; Kazantzis, Deane, & Ronan, 2005) is a 12-item rating scale which will be used.

            The video segments will be recorded role-plays of cognitive behaviour therapy with the primary investigator (as therapist) and an actor (as client). In each segment, the therapist and client will review the client’s level of engagement with the homework task over the previous week in a standard duration of 5 minutes. The content of the segments will include standard questions regarding the review of homework assignments, such as “how did you find the task we talked about from last session?” and “was it difficult and/ or easy for you?” These segments will be focused on the practical aspects of task completion, and the therapeutic content will not be discussed in detail.

Procedure

Participants will be involved in one 40 minute data collection session. The study will be run in lecture halls allowing for up to 100 individuals to participate simultaneously. Participants will be asked to arrive and be seated in the lecture hall shortly prior to the commencement of the session, according to which sequence of video segments they are to view. Participants will be seated in rows (at random) that will signal their allocation to information groups. Participants will be welcomed to the study and asked to read the information sheet, which will provide a brief rationale for independent observer ratings of therapy sessions, and the nature of the present study. Participants will be informed their attendance in the session implies consent.

The experimenter will read standardised instructions detailing the scenarios to be displayed in the video segments. The instructions will make clear that video segments will only be viewed once. Participants will have 5 minutes to review the questionnaires. Participants will then be given a brief scenario using a PowerPoint display, and asked to practice completion of the questionnaire from an observer’s perspective. Following this practice, participants will be invited to ask any questions to clarify the content or process of completing questions.

It will be stated that participants should concentrate on the video segment, and once viewed, complete the 12-item Homework Rating Scale – Revised (HRS-II; Kazantzis, Deane, & Ronan, 2005). After completing the first rating, participants will be presented with either (a) no further information, (b) client HRS-II rating, (c) therapist HRS-II rating, or (d) client and therapist HRS-II ratings, and offered the opportunity to change their responses. Participants will be instructed to use circles for their initial responses, and squares for their second responses on the same questionnaire. After completing the first rating there will be a short 2-3 minute break when forms will be collected, and participants will asked to remain silent. After the break, participants will complete the second rating following the same procedure.

After all the video segments have been rated, the experimenters will remind participants that the scenarios were role-play segments and not actual clinical sessions. The experimenters will invite any questions or feedback from the participants. Participants will receive the results of raffle via email and the winning participants will be asked to collect their prize. Participants will be offered the opportunity to obtain a certificate of participation.

Methods of data analysis

The effects of the between-groups and within-groups factors will be calculated and analysed using multivariate analysis of variance (MANOVA). MANOVA will be used with Pillai’s trace as the criterion, as opposed to repeated measures analysis of variance (ANOVA) because of the multiple measures involved. The data will initially be screened in preliminary tests to rule out age and gender effects and test any outliers or unusual variations in the data that had not been considered. The counterbalancing procedure for session content will be considered effective if there is no main effect for session sequence.