Chapter 3

 Methodology

3.1 Participants

Ethical approval from Queen’s University, Belfast was obtained before the outset of the research. The study followed the standards defined by the Behaviour Analyst Certification Board in the Professional and Ethical Compliance Code for Behaviour Analysts (BACB, 2014) for scientific competence and ethical research. Through the delivery of consent forms, permission was obtained from participants’ parent/guardian before the research was undertaken. The consent form consisted of; acknowledging potential benefits and risks related to taking part in the study; how data will be collected, stored and disposed of; how data will be anonymized and kept confidential; at what point in the study and how consent could be withdrawn; the researcher and the researcher’s supervisor’s contact information. The consent form was sent home with the children from school prior to data collection. There is a copy of the consent form available in Appendix (chapter 8).

The participants in this study were three children diagnosed with ASD attending a special education needs school in the north of Wales providing for approximately 200 pupils between the ages of three and 19 years. All participants attended the school Monday-Friday 9:00-15:00 and were eligible for the study as they engaged in frequent off-task behaviour that interfered with learning.

The requirement for inclusion in the study was a lack of on-task behaviour displayed by the student during independent work. To demonstrate on-task behaviour, students must 1) actively listening to teacher instructions. An example of this is by responding verbally (e.g. answering questions) or responding nonverbally (shaking head). 2) Abiding by the teacher’s instructions, 3) facing the teacher or task and 4) seeking help appropriately e.g. asking questions or raising their hand (Allday and Pakurar, 2007). Non-examples include: looking out of the window, not looking at the teacher, holding and playing with items, head down on /facing the desk for more than 5 seconds. The three students that participated in the study were chosen by the teacher for displaying the least on-task behaviour during independent work in the classroom.

Neither participant was taking medication during the course of the study. All three were members of a class of ten children with one teacher and three teaching assistants (0.4:1 staff to child ratio). The school is bilingual (English-Welsh) therefore work and materials in the classroom are all presented in both languages in accordance to the 1993 Welsh Language Act which requires that all programs, assessment and training materials are presented bilingually (Welsh Language Act, 1993). Working alongside the teacher’s in this school are an ABA team. The team consists of a consultant behaviour analyst, a BCBA and two assistant-level behaviour analysts. The team work with the teachers to create and carry out behavioural interventions or plans that are based on the principles of the science of ABA. They also provide training for teachers and classroom assistants to deliver the teaching programs and behavioural interventions.

Participant 1, who for the purpose of this investigation will be referred to as Owen, was seven years old at the onset of the intervention and had been attending the school since he was five. Owen was eligible for this study as his teacher described that he frequently engaged in off-task behaviour in the classroom through echolalia, or distracting his peers through touching them, or talking to them.

Participant 2, Sara, was seven years old at the beginning of the intervention and had only just moved from a mainstream school in September 2016. Sara consistently displayed self-stimulatory behaviour in the form of hand flapping during work sessions, and frequently got up from her chair to walk around class.

Participant 3, Daniel, was eight years old at the beginning of the intervention and had been attending the special needs school since he was five. Daniel’s teacher noted that he had great difficulty staying on-task in the classroom as he often looked out the window, and sat very low in his chair in a lying down position.

3.2 Setting

All observations and intervention took place during work sessions the classroom on normal school days for a period of 8 days (2 school weeks). This isolated room was familiar to all the children and consisted of two tables with ten chairs, a whiteboard at the front of the class where the teacher stood, and three cupboards at the back of the class where the children kept their bags and teaching materials. The day was split into periods as follows;

Table 1. Normal school-day routine for participants’ class

This schedule was strictly followed in order for the pupils to learn their routine and be comfortable in school.

3.3 Materials

The MotivAider was used as a tactile cue to self-monitor throughout the intervention phase as the independent variable. The pocket-sized device can be effortlessly programmed to vibrate at different time intervals with a range of 1-5 strength of vibration option. The MotivAider was attached to the child’s waistband and programmed to vibrate on a 3-minute fixed-time schedule with a vibration strength of 4. The MotivAider would automatically re-start after every vibration. The student was presented with a self-monitoring form, which is available in the appendices, to record whether he was on-task at the time the MotivAider vibrated.

Digital timers to measure duration of observations were used, data sheets appropriate for event recording and for general note-taking were designed for the needs of the study, the FAST questionnaire to determine function of behaviour, and a social validity questionnaire for the teacher were used and can be found in the appendices.

4. Functional Behaviour Assessment

A Functional Behaviour Assessment (FBA) was conducted for all three participants using an indirect assessment method; The Functional Assessment Screening Tool (FAST) (Iwata and DeLeon, 2006) to identify the maintaining variables of lack of on-task behaviour. The FAST (Iwata and DeLeon, 2006) is a 16-item questionnaire concerning antecedent and consequent events of an episode of problem behaviour. The person completing the questionnaire is asked to circle a ‘Yes’, ‘No’ or ‘N/A’ for 16 statements. It is organized into four categories based on contingencies that maintain problem behavior; 1. social-positive reinforcement (access to attention/tangible items), 2. Social-negative reinforcement (escape from demands), 3. Automatic-positive reinforcement (self-stimulatory behaviour) and 4. Automatic-negative reinforcement (removal of pain/discomfort) (Iwata, DeLeon and Roscoe, 2013). For the remainder of this paper the four categories will be labelled as follows;

Social Positive Reinforcement = Attention

Social Negative Reinforcement = Escape

Automatic (Positive) Reinforcement = Sensory Stimulation

Automatic (Negative) Reinforcement = Pain Attenuation

The number of ‘Yes’ responses are counted and analysed in the scoring summary section of the questionnaire. 1-4 ‘Yes’ responses denote that the behaviour is maintained by Attention, 5-8 ‘Yes’ responses denote that the behaviour is maintained by Escape, 9-12 responses suggest a Sensory Stimulation function and 13-16 ‘Yes’ responses suggest the behaviour is maintained by Pain Attenuation.

The decision to use the FAST as an assessment tool was based on its ease of use, its unobtrusive nature, convenience in that it would not disturb the participants’ daily routine in school, and its time-saving feature. The FAST was completed independently by the teacher and then discussed in an informal interview. This was conducted to evaluate whether the use of the MotivAider to increase on-task behaviour would benefit children with different functions of off-task behaviour.

3.5 Reinforcement

When a behaviour is followed by reinforcement, that behaviour will appear more frequently in future (Cooper, Heron and Heward, 2014). A reinforcement system for work sessions was already in place before the beginning of this study, and was used during the research in line with their usual routine. Before the beginning of every work session in the classroom, the children are presented with their ‘choosing board’. These ‘choosing boards’ consist of pictures of their favourite toys or activities e.g. iPad, outside play, chocolate buttons etc. Parents and members of staff are continuously interviewed during the school year in order to gain a pool of potential reinforcers for each participant. The pictures of these toys and activities are stuck on the board using Velcro. The children use the boards to choose what they would like to work for by pulling off their desired picture and handing it to their teacher. Materials and toys from each child’s ‘choosing board’ were used as reinforcement at the end of each work session.

3.6 Measurement:

Momentary time sampling was used to measure the dependent variable on-task behaviour at 20-second fixed intervals during an observation period of 10 minutes. The observation period took place during work sessions in the child’s daily schedule. If the student presented on-task behaviour at the end of the interval the student’s behaviour was recorded as on-task for that interval. The dependent variable, ‘On-task behaviour’ was defined as follows:

To demonstrate on-task behaviour, students must 1) actively listening to teacher instructions. An example of this is by responding verbally (e.g. answering questions) or responding nonverbally (shaking head). 2) Abiding by the teacher’s instructions, 3) facing the teacher or task (reading or writing on their worksheet (Holifield et al., 2010)), and 4) seeking help appropriately e.g. asking questions or raising their hand (Allday and Pakurar, 2007). Non-examples include: looking out of the window, not looking at the teacher, holding and playing with items, head down on /facing the desk for more than 5 seconds.

3.7 Research Design and Procedure

An ABAB reversal design was used with the three participants. Phase A (baseline) was conducted to obtain information regarding pre-intervention levels of on-task behaviour. This was followed by intervention with implementation of the tactile prompt (phase B). Then a return to baseline (A) was conducted and finally a second intervention phase (B). A training phase was implemented after the end of the first baseline period.

The baseline establishes the performance of the participant in the absence of the independent variable, in other words, in the absence of the intervention. This acts as a basis for detecting the effects of the independent variable when it is introduced in future. The goal of a baseline is to establish a stable and predictable trend prior to introducing treatment. A stable and predictable trend of responding can be defined as a pattern that “exhibits relatively little variation in its measured dimensional quantities over a period of time” (Johnston and Pennypacker, 1993a, pp. 199). If the baseline trend is ascending or descending in the therapeutically desired direction, or if the baseline is unstable, the independent variable should be withheld. If the behaviour begins to deteriorate, or the baseline stabilises, the independent variable can be applied (Cooper, Heron and Heward, 2014). During this study, baseline was measured until stable, prior to the introduction of the intervention.

Baseline: During baseline, self-monitoring procedures were not in place. Teachers were instructed to use their typical classroom management procedures for the target behaviour. The child was observed during work sessions with staff: child ratio of 1:4 or 1:5. The researcher observed the child from a distance that would allow them to identify the target behaviour, while being as discrete as possible to reduce reactivity effects.

Training phase and video modelling: Each student was trained to self-monitor his/her on-task behaviour in a video-modelling and a practice session. During the video-modelling session, the students were presented with two short videos to provide visual examples of the target, and of non-examples of the target behaviour. The first video showed what consisted of on-task behaviour and lasted approximately one minute. The video showed a 12 year-old boy sitting at a desk, looking at and writing on the piece of paper in front of him. He then feels the MotivAider vibrate on the waistband of his trousers and notes on his self-monitoring sheet that he is on-task.

The student was also taught what did not consist of ‘on-task behaviour’ in a second video which lasted approximately one minute.  The second video showed the same boy sitting at a desk, with a piece of paper in front of him. The boy looks out of the window when he feels the MotivAider vibrate on his waistband and notes on his self-monitoring sheet that he is not on-task.

Each student received one training session prior to the intervention which lasted approximately 10 minutes. During the session, there was an introductory stage aiming to ensure the vibration of the pager did not cause distress to the child. The MotivAider was placed on the child’s hand to feel the vibration. Following this, the MotivAider was attached to the waistband of the child.

During the practice session, the child was asked to colour in a worksheet and the MotivAider was attached to their waistband. On the vibration of the MotivAider the child was required to fill in the self-monitoring sheet and return to work. The mastery criterion for each step of this stage was two consecutive correct responses. Both practice and training sessions took place in an isolated room outside the classroom.

Self-Monitoring Intervention: At the beginning of the session, the teacher presented the student with his/her choosing board and instructed the child to choose what they wanted to work for. After this, the MotivAider was attached to the child’s waistband and the teacher continued to teach as usual. When the MotivAider vibrated, the student noted whether he was on-task on the provided sheet. The work session staff to child ratios were 1:4 or 1:5. During the work session, the children were given work to complete using a 5-10 minute instruction phase, followed by independent work. Data was collected during the independent work phase for 10 minutes across a variety of different activities including maths, literacy and art.

The person(s) collecting data observed the child from a distance that would allow them identify the target behaviour while being as discrete as possible. The data collector sat behind and across from the child.

Additional training was required for Daniel as he would take the MotivAider off his waistband and hold it in his hand to look at it. This distracted him from his work. When this occurred, the behaviour was ignored and MotivAider placed back on his waistband. Daniel received the training phase prior to every work session when he wore the MotivAider.

3.8 Validity

Social Validity: Even if treatment is found successful of effective by researchers, if it is not found useful, helpful, productive and socially valid by the significant others of the participants e.g. parents, teachers, carers, then it is not socially valid (Horner et al., 2005; Horner et al., 2012). Wolf (1978) recommended that data should be collected to identify the values of the intervention to society in three ways; the social significance of the target behaviour, the appropriateness of the procedures, and the social significance of the result e.g. would participants consider using the intervention outside the study, or if they would recommend the intervention to others in future. This is to “help choose and guide program developments and applications” (Baer and Schwartz, 1991, pp. 231), which is ultimately the purpose of social validity assessments (Cooper, Heron and Heward, 2014). Therefore, following the final intervention phase, the classroom teacher completed a social validity questionnaire. Items were rated on a Likert-type scale designed to allow respondents to rate their level of agreement or disagreement with a series of statements. These levels of agreement were as follows; strongly agree, agree, disagree and strongly disagree. The assessment consisted of statements regarding helpfulness of the intervention in the classroom, future use of the intervention, and possible benefits for both students and teachers from the intervention. The questionnaire was completed in less than five minutes and can be found in the appendices.

Internal Validity: Experiments demonstrating high internal validity are experiments that demonstrate clear experimental control; that the changes in the dependent variable, the behaviour, are a function of the independent variable, and not from uncontrolled variables (Cooper, Heron and Heward, 2014).

An ABAB reversal design was used in order to demonstrate experimental control in the form of verification and replication. When the intervention is removed, verification is affirmed as the baseline measures return to initial levels. This confirms that the behaviour is changed by the intervention, and would remain unchanged without the intervention. When the intervention is reintroduced and an increase in the target behaviour is presented, replication is demonstrated. An ABAB design is preferred to an ABA design as it strengthens the demonstration of experimental control because it reintroduces the treatment phase which enables the replication of treatment effect.

Confounding variables minimized to increase internal validity:

Maturation: Changes can take place in a subject over the course of time that can be a confounding variable e.g. improved performance may be the result of the acquisition of new skills. This study used rapidly changing conditions and multiple introductions and withdrawals of the independent variable to control for maturation.

Observer Training: A potential threat to reliable data is inadequate observer training. Observers must be able to differentiate between an occurrence or non-occurrence of the target behaviour and record accurately on a data sheet. In this study, the researcher and inter-observer agreement (IOA) data collector received training via video modelling of how to recognise and record the target behaviour. This training was provided to the IOA data collector prior to every work session in order to minimize observer drift.

Observer Reactivity: In order to reduce the participants being aware that they were being observed, the person(s) observing the participants did so from a distance that would allow them identify the target behaviour while being as discrete as possible e.g. sitting behind and across from the participant. Additionally, to reduce the influence of observer reactivity in the data collection, the person(s) collecting data sat on different sides of the room. If one observer predicts that the second observer will record the behaviour of a specific interval in a particular way, the data could be influenced by what the first observer anticipated the other observer may record. By placing both data collectors on different sides of the room, it minimizes the potential for them to consult each other during data collection, and focus entirely on the target behaviour.

External Validity: External validity is the degree to which the functional relation is found reliable and socially valid under different conditions. To maximize external validity data was taken during several activities (maths, literature, art) and across several times of day (morning, mid-day and afternoon work sessions).

3.9 Fidelity of implementation and inter-observer agreement (IOA)

Prior to the beginning of the study, the rationale, procedure and materials of the project were presented to the classroom staff, following an opportunity to ask further questions. On-task behaviour alongside non-examples were demonstrated via the video-modelling that the students received.

The researcher served as the primary data collector, and a Postgraduate Certificate of Education (PGCE) student on work placement in the school collected inter-observer agreement (IOA) data for 20% of the sessions. For the purpose of IOA the second data collector was further trained to identify on-task behaviour by re-watching the video model of the target behaviour before each data collecting session. IOA was collected during the first and third day of data collection across two participants. IOA was calculated by dividing the total number of agreements, by the total number of agreements and disagreements, multiplied by 100. On average, IOA was 96% (range 92% to 100%).

Chapter 4

Results

4.0 Functional Behaviour Assessment:

An indirect functional behaviour assessment was completed by the teacher in the form of the Functional Assessment Screening Tool (FAST) (Iwata and DeLeon, 2006). The scoring criteria is summarised in the table below and is taken from the FAST which is available in the appendices.

Table 2. Scoring Summary for the FAST

The results are summarised for all 3 participants and are available in Table 3 below.

Table 3. Results of Functional Analysis Screening Tool for each participant’s decreased on-task behaviour