Research Methods BT1 Numeracy

Abstract

This research will investigate the readiness of student’s to join the basic training program. It will focus on the perceptions of staff at No1 Radio School while marrying these comments to student performance. The research will focus on the levels of numeracy within the student population and their ability to apply this to the course material, whilst relating this to their current level of qualifications. These issues are not purely associated with the Royal Air Force’s training but are prevalent throughout the education and training sectors. The subjects of literacy and numeracy have been identified as areas that are vastly under research, it is hoped that the issues raised in the short research project will identify further research questions that in turn may form part of further research.

Introduction

The field of work this research will investigate lies within the MoD, specifically the RAF at DCAE Cosford. The school being used to conduct this research is No1 Radio School, delivering training to Basic Trainees and Assimilation Trainees for all ground trades now known as Trade Group Four (TG4). The student’s ages range from 17 to 30 years of age. This research will focus on one specific target group, Basic Training 1 (BT1). These students are new members of the Royal Air Force and will have just completed their basic training at Halton (Ten weeks); they have come to Cosford to begin their trade specific training.

The course is ten months in length, over which the students will cover both principles and trade training. The entry requirement for this trade is a minimum of three GCSEs, in Physics, Maths and one other subject. However they do accept some individuals with lesser qualifications as an exception and occasionally current members of the RAF who wish to change their trade. The trade requirements are specific and therefore it would be reasonable to assume the students basic level of literacy and in particular numeracy, would be sufficient to allow them to develop throughout this course.

Research Issue.

This research will look specifically at the levels of numeracy within the BT1 student population. It will try and identify if there is a problem and seek to provide evidence of any issues while providing recommendations for improvement. To ensure that there is no breach of ethics, the research will focus on the perceptions of the staff and the results from the students.

This research issued raised questions that required answering, ‘why is it that qualified students find the level 2 key skills and the level 2 maths phase of the BT1 training difficult?’ It is these issues that have intrigued me for some time.

I believe this issue is worthy of research, however it is quite possible that additional questions will be raised that fall outside the scope of this research and may require additional research before they can be answered.

Research Focus

It has become apparent that new students entering the Royal Air Force as trade group 4 operator maintainers, although qualified for trade do not appear to posses the prerequisite knowledge required to complete the Key Skills at level 2 and the Maths phase of the course. That is not to say they all fail. That would be far from the truth, it is the difficulty they demonstrate when completing this phase of the course. The reason that this research is so important is this first phase of the course is crucial for the students. It gives them the skills and knowledge they require, which in turn will allow them to apply this knowledge to new problems and situations as they progress through the course. These skills could be considered tools of the trade, without a good grasp of these concepts the students will not gain the maximum benefit from the course.

Literature

Literacy and numeracy have become major issues in the field of education since the Moser report 1999, the working group chair by Sir Claus Moser in the opening statement of the report stated,

• Something like one adult in five in this country is not functionally literate and far more people have problems with numeracy.

Moser, C (1999)

This statement is met with surprise when you discuss this subject, however Moser states,

• We have found that people are staggered when one confronts them with the basic facts about literacy and numeracy, and rightly so…It is a state of affairs that cannot be allowed to continue, and our Report proposes a wide ranging approach to the challenge

Moser, C (1999)

Many organisations have concerns about the statistics when it comes to Literacy and numeracy in the adult population, Wells, A (2004) states,

• Surveys in the 1990s suggested that between 15 and 19 per cent of adults had poor basic skills. However, the International Adult Literacy Survey in the late 1990s suggested a much higher figure of 23 per cent or seven million adults.

Wells, A (2004)

Are we seeing a true picture of the problem? This research has highlighted concerns from staff about the student’s abilities. It would seem from the statistics that there is a strong correlation.

One area this research was unable to quantify was how does gender affect numeracy, It was not possible to confirm or deny this because there were insufficient females included in the sample group to make a good judgement. However a recent article published by the DFES suggests there is a considerable difference between females and males when it comes to numeracy. Grinyer, J (2006) states,

• Women and men perform near equally for literacy, but men significantly outperform women at numeracy.

Grinyer, J (2006)

At level 2 this is an interesting fact and one that is perhaps worth of further research. If gender is an issue then what about current qualification level. Grinyer, J (2006) states

• Highest qualification appears to be the best predictor of an individual’s literacy and numeracy level, with parental background and local deprivation having larger effects than current employment status

Grinyer, J (2006)

Research Hypothesis.

The research hypothesis should be constructed in such a way that the research can disprove the hypothesis. So while the researcher believes students have issues with numeracy that directly affect their ability to perform throughout the course, and that the accepted entrance qualifications standards do not reflect the abilities of the students, the hypothesis should refute this. (See the hypothesis)

The Hypothesis for this piece of research is:

All BT1 trainees are sufficiently numerate to attend the BT1 course, as all trainees are qualified to a minimum standard. (Three relevant GCSE’s)

Research Hypothesis.

There are a number of possible questions encompassing a range of different methodologies the scope of this research will be limited to one Research question, below are two possible questions.

• ‘Why do students on the BT1 course suffer with the issue of numeracy when attending the taught lessons, completing assignments and examinations when they are course qualified?’

• ‘How do staff perceive the readiness of students on the BT1 course to deal with the academics of the course?’

After consideration and due to the constraints of this research it was decided to use the later, an interpritivest approach. Using this question offers the research some distinct advantages, firstly a limited scope and secondly and more importantly, it eliminates the students from the research which removes a large percentage of the ethical issue that may arise from questioning the students directly.

This research should identify how instructors feel about the perceived problems students have when using the different forms of numeracy.

• Mercer, N states, “one function of theories is to set agendas for research ¿½ to generate certain kinds of questions which the research will attempt to answer.”

Mercer in Walford (1998 p.42)

This question could use a likert style method of answering, thus allowing the individual to grade their agreement or disagreement of the above question. This question as it stands would not necessarily be suitable for the students to answer as it is likely to produce a Yes/No response, however it could be acceptable for the staff. The same question could be asked in a different way. i.e. How has the GCSE maths helped the students with their study of the other principle subjects?

Although it would be more difficult to analyse a short answer would bring out more of the individuals feelings, where as the likert style just grades the agreement or disagreement. Other questions that might be asked are:

I. Do you feel the students could complete this course successfully without this qualification (GCSE maths)?

II. How did the student’s maths affect their ability to complete this phase of training?

III. Has the student qualification helped them with their studies?

Youngman (1986) in Bell said, “The more structured a question the easier it will be to analyse.”

Bell (1999 p.119)

Research Paradigm

What is a paradigm? A Paradigm ¿½ group of coherent ideas about the nature of the world and of the functions of researchers that are adhered to by a group of researchers, conditions and patterns of their thinking which underpins their research actions.

What paradigms exist? There are a few key paradigms that exist within educational research positivist and Interpretiveist. Positivist research uses scientific, Statistical analysis and empirical methods,

• Cohen and Manion said, “Traditionally, the word refers to those techniques associated with the positivistic model ¿½ eliciting responses to predetermined questions, recording measurements, describing phenomena and performing experiments”.

Cohen and Manion(1996 p.38)

Interpretiveist research deals with human emotions, personnel perspectives.

• Cohen and Manion said, “The central endeavour in the context of the interpretive paradigm is to understand the subjective world of human experience.”

Cohen and Manion(1996 p.36)

Ethnographical methods are interpretivist. Gall, M. et al (1996, p607) say, “Ethnography involves first hand, intensive study of the features of a given culture and the patterns in those features.”

Judith Bell’s thoughts on quantitative and qualitative research identify that quantitative researchers collect facts and study the relationship. They are measured using scientific techniques producing quantifiable and generaliseable conclusions, where as a qualitative perspective is more concerned with individual’s perceptions of the world; Seeking insight rather than statistical analysis. In addition she concludes that on occasions qualitative researchers draw on quantitative techniques and vice versa. This would suggest that one method alone does not have all the answers and therefore we could consider various methods when conducting research.

This research will use an interpretiveist methodology, this approach has been selected as the most appropriate as the information will come from members of staff, the information they provide will be their interpretation of the situation hence interpretiveist. This approach has been chosen as more appropriate than positivist approach because there is little scientific evidence and the amount of data collected would be insufficient to produce any valid statistical analysis. Ethnographical and eclectic research was also dismissed although it bridges the gap between Positivist and Interpertiveist research. It would however,still have positivist elements as mentioned previously. However the over arching reasoning for adopting an interpretiveist approach was the limited nature of this research project. Although, it is hoped that the findings from this research will generate further questions and spark yet further research questions for future research

Research Methodology

This style of research could be considered a case study, as its focus is centred on a specific group of students, belonging to a single organisation. Cohen and Manion state, ” a case study researcher typically observes the characteristics of an individual unit ¿½ a child, a clique, a class, a school or a community”

Cohen and Manion (1996,p106)

While this approach would work it is more suited to a continuous improvement program and not a specific piece of research, in addition the length of time it would take to conduct a program of this nature makes this approach unsuitable.

In planning the research, various methodologies were considered, however due to the limited nature of this research and to limit the impact of ethical issues, this research decided that an Interpretiveist approach would be most suited. This style of research is relevant, as the research will investigate the attitudes and opinions of the staff and not the students, while using student data to support or refute the findings.

To facilitate this research there were a number of preferred techniques available for gathering the data:

• Interviews / Meetings
• Academic results
• Reference to external sources (Collages, Universities, Schools, Government Departments)

The main reason behind this strategy, methods and analysis tools was time. The other key issue was ethics, for these reasons this research will be conducted through focus group meetings with staff using structured questions, this data will then be compared with the average results for a class. This approach will limit the impact of ethical issues and allow some conclusions to be drawn. This approach was discussed with the staff that felt that due to busy teaching commitments this was the most appropriate solution.

• Judith Bell states, “Talking through problems and possible topics with colleagues is an essential stage of any plan.”

Bell (1999 p.21)

The focus group will consist of five lecturers and four instructors, it is envisaged that a single meeting will take place to pose the questions to the staff. The data will be collected via questions directed to the staff in discussion groups; the responses of the staff were recorded. (See Annex B) To support the findings from the discussion groups the student’s results will be analysed.

Data Collection instruments

The data collection will be achieved through structured questions delivered at group meetings where the responses will be tabulated; this will give a feeling for the staffs’ concerns and perhaps some misconceptions. It will also allow any further research to focus its attention on specific areas of concern. Once the data has been collected this can be cross referenced to the student results, identifying any disparities, in addition student results and student retention rates may be useful sources of information.

The other methods considered were, interviews, these could be time consuming, as staff are not always available. A well constructed questionnaire could produce valid reliable results in a reasonable time frame. However due to the limited time constraints, this research will utilise meetings with a number of groups and evaluate comments from those meetings to produce an Interpretivist style report. This will not provide hard evidence, however it will illustrate any concerns or trends that may be apparent which in turn could instigate further research. This approach although less structured will give a general feel for the issues, which could lead to further research questions.

Sapsford and Jupp note the following facts about collecting data, “There are both advantages and disadvantages to both highly structured and less structured methods; in no sense is it true to say that one is more objective than the other.”

Sapsford and Jupp (1996, p94)

Research validity ensures the data collected is accurate and meaningful, collecting data in a non valid way would affect the final analysis; which may produce misconceptions when analysing the research data, thus rendering the research valueless. The research will improve validity through accurate data collection and careful analysis of the results, in addition reliability and repeatability could be enhanced through the triangulation of the data collected. Triangulation is method of verifying the data, checking and ensuring accuracy and ensuring that the findings are qualitative and thus reducing the chance of unsubstantiated data.

• Cohen and Manion said, “Triangulation may be defined as the use of two or more methods of data collection the study of some aspect of human behaviour”.

Cohen and Manion(1996 p.233)

The triangulation of the data was achieved through data collected from the meeting notes; these were then supported by student results and reference to national statistics.

Chosen methods of data collection.

This project will use a structured set of questions derived from the research question. The aim was to ask these questions at the focus group meeting, where the staffs’ responses and opinions were recorded. The rationale behind the choice of research and the methodology is predicated upon the limited nature of the research this in turn has limited the choice of methodology (interpertiveist).

The limited time scale in which this research was conducted has in turn restricted the modes of research used to produce the data for evaluation; the aim was to gain a feeling from the staff as to how they perceived the research issue. The use of structured questions helped guide the process of data gathering and allowed the collection of data within a very short time scale, this was imperative due to the limited time constraints of this research. The interpritiveist approach allows the staffs’ feelings to be represented against the set criteria thus giving a good degree of relevant data for analysis. While this approach aided the collection of data at a rapid rate it did not offer the widest selection of data. Other methods could have produced further data to support or refute the findings. However this research hopes to triangulate its findings against student data (examination and assignment results).

Research ethics.

The matter of ethics is an important one; if the researcher wants to achieve open and honest answers they must consider ethics when conducting the research. To reduce the impact of this research the methodologies have been carefully chosen to reduce the data collection to that of the staff. The students’ results will be anonymous, to such an extent that the research will not identify the individual students, thus reducing the opportunity for others to identify individuals through the results and class group.

Researchers cannot always predict what they might find and they may not like what they find; however it is the duty of the researcher to be open and honest about the research they are conducting. These are some of the things to consider when conducting research: People’s amenity, that is to say protecting an individual’s identity, ensuring that a respondent’s comments cannot be tracked back to a single individual or group.

The Oxford Brookes University classifies research ethics and says,

• “The integrity of any research depends not only on its scientific rigour, but also on its ethical adequacy. Ethical issues are many and varied, and may be quite complex. Research involving human participants is undertaken by many different disciplines and conducted in a broad range of settings and institutions. While some issues are specific to professional groups, all research should be guided by a set of fundamental ethical principles to ensure the protection of human participants.”

Wood,L (2006)

Design Constraints

One of the main issues when conducting any form of research is time; given enough time the research could produce massive amounts of facts, figures and an impression of people’s feelings. In addition to time there is always the cost of conducting research. However most organisations would expect any form of research to be conducted within certain time constraints, thus limiting costs and obtaining results within that time frame. Therefore the research can only be expected to deliver some of the information that is available, and that data would need to be substantiated through triangulation, generalisability and reliability. It is the limited nature of this research that has influenced the chosen methodology, interpertiveist. Although this may limit the validity of the research, the triangulation of the staffs’ comments and answers to the set questions when linked to student results, should give a reasonable confidence in its findings.

Analysis of Results

The questions asked at the focus group and the responces are given below:

The group was asked, Are current BT 1 Trainees academically equipped to join this course?

The staff responded with the following statements:

• Although students are qualified they do not seem to be prepared for the challenges this course offers.
• Some staff feel that GCSEs are a lower level qualification than a traditional ‘O’ level (This used to be the entry requirement some years ago.)
• Although the students are qualified they do not seem to have the prerequisite skills to work effectively on the course.
• Students do not seem to remember some of the basics of mathematics.

It would appear from the response that the staffs’ perception is that students are not prepared for this course and that the qualifications they hold do not reflect the student’s abilities. The BBC news web site on 30 October 2003 stated,

• The survey found 47% of the adults in England or 15 million people had a lower level of mathematical knowledge than was needed to gain a grade G at GCSE. The Department for Education and Skills blamed
• “decades of neglect” for figures showing millions of people lacking basic literacy and numeracy skills.

http://news.bbc.co.uk/2/hi/uk_news/education/3227263.stm

There seems to be a general distrust of current qualifications as opposed to the traditional ‘O’ Levels. This was the minimum entrant standard for this trade.

The group was then asked, Which area of the course do students find difficult?

• The Maths is the most difficult for students because they do not possess the basic skills required to complete the work.
• Key Skills, Numeracy
• Some struggle with Electronic fundamentals mainly due to a lack of maths ability.

They coped well with Addition and subtraction, however multiplication, division were more problematic. Additional areas where students struggled were percentages, area and fractions. These are not the only subject areas where students found difficulty, however they cover the major problem areas. These issues are similar to those identified in a recent TES, they stated,

• In last year’s skills action plan, the Assembly government revealed that 24% of the working age population lacked level 1 literacy skills and 53% level 1 in numeracy. In Wales, 20% have no qualifications compared with 14% across the UK

(TES Cymru, June 30 2006)

The focus of the problem seems to be centred on the Maths / arithmetic elements of the course. One of the key areas was Key Skills, in particular numeracy. Students seemed to find this difficult; they did not seem to posses the basics.

Staff were asked to consider, Are there any circumstances that might affect students’ ability to perform on the course?

• Poorly prepared when leaving school.
• Length of time away from a learning environment.
• Lack of ability
• Some seem unable to apply previously learnt knowledge to new situations.

Its is difficult to believe the students were poorly prepared as they all are qualified, suggesting they once had the knowledge, however it is interesting to note the comment about the length of time, as it is well known that constant and repetitive use of knowledge aids retention, lack of use leads to little or no retention of knowledge.

A recent news article in the Daily Mail by HARRIS, S (2006) stated, “One in five about to embark on teaching careers still have problems spelling and using punctuation as well as doing the most simple multiplication, division and percentages.”

Harris, S (2006)

The question, Do you believe the students are suitably qualified?

Staffs’ response to this was, Yes, they are for the trade and on paper, but do the qualifications match up to the expectations of the course?

This alludes to a previous question about students being prepared. Do the qualifications meet the minimum requirements of the course. The answer is yes they do, however there seems to be a void between students retained knowledge and the paper qualifications. To illustrate this issue the House of Commons raised the issue of literacy and numeracy with the Secretary of State for Defence being asked, “how many and what proportion of new recruits to the army in the last 12 months had (a) literacy and (b) numeracy skills at or below (i) entry level 3, (ii) entry level 2 and (iii) entry level 1.” Mr. Touhig replied, “Consolidated literacy and numeracy initial assessment figures for recruits to the army in the period 1 November 2004 to 31 October 2005, are provided as follows.”

These figures were given from an Official Report, column 98W on departmental staff literacy and numeracy.

No Author (2005)

The group was also asked, Are students prepared for the academic rigor of this course?

• No, Students are ill prepared when it comes to learning in this environment.

This may also relate to the time away from a learning environment, the beginning of the course is quite intensive and therefore the students find it difficult to get back into a learning environment.

In summary of the responses, the staff feel the students were inadequately prepared for this course. However this was not for lack of qualifications more a perceived difference between the more traditional ‘O’ Level and the existing GCSE qualifications, while the time away from the learning environment may have its part to play.

Student results for the first eight classes illustrate that not all students struggle to achieve good results. (RAF pass is set at 60) So could this purely be a staff issue and not one of the students. As the results below indicate some classes obtain reasonable results, that is to say, not all struggle.

The results for the first eight classes are shown below:

Maths

Electronic Fundamentals

While these classes passed the respective subjects, there are some low scores in maths for classes 1, 4 and 8. Electronic Fundamentals produces better results, however class 1 and 7 have still scoured low marks. This information is not entirely conclusive as can be seen from the results, some classes did well.

Findings

The hypothesis has been refuted through the data collated from staffs’ responses to questions and the student data, all be it limited in nature this research has been shown to support the staffs’ comments.

The analysis of the questions suggests a lack of ability on the student’s part, this however does not seem to be predicated on qualification. The staffs’ perceptions of students abilities seem to suggest the students are not sufficiently prepared to attend this course, more over they are not used to the training methods utilised within the MOD. These facts when grouped together cause the students and staff some problems, however this is not to say that large numbers of students are failing to achieve a pass in these subjects, therefore is there a problem?

On the face of it, no. However modifications to the training methodology and some additional sessions may improve some of these issues.

This research has investigated a broad range of issues relating to new students entering the RAF’s BT1 training program, their ability to cope with the academic pressures of the course and their ability to complete the work to the required standard.

It would appear many students, although qualified on paper have some difficulty when it comes to the Maths and Electronic Fundamentals phases. Some of the underlying problems relate to the methods of delivery on the course as it is unlike traditional academic institutions. This means that the students take time to adjust to this training methodology and as the Maths and Electronic Fundamentals phase of the course is at the front end, it does not allow sufficient time for this adjustment to take place.

On the subject of time between learning, many students left school twelve months before joining the course and have not practiced these skills for some time. As with the maths unless a student continues to use these skills they are quickly forgotten. However this does not explain why students do not remember when prompted. Perhaps this could point to a lack of understanding at the initial point of learning and may require further research to substantiate this theory.

Recommendations

This piece of research has been limited in its nature, however it has allowed the researcher to gain a wider perspective on these issues, to further improve this work and make the findings more valid it is proposed to include the students in any future research. The students were excluded in this case to speed up the data collection and to reduce any ethical issues that may arise from the research. Further research could be developed from the fi