How to tackle Evaluation and Analysis in the RAP Part 2

[If you haven’t read Part I it may be appropriate to briefly read it through as some of the general comments may be applicable to other topics]

How to Approach Evaluation in Connection with Topics using Primary Data Collection.

First the most important aspect of using primary data is in designing your data collection. So please read Phil Clarke’s article on this (pp36-38 of the Information Pack). Then remember the phrase “Garbage in, garbage out” – in other words if your data is faulty so will any results that you try to infer from it. This is likely to result not just in a fail for Evaluation, Analysis & Conclusions (the data is unreliable so the resulting analysis and conclusions are invalid) but also in Application (the poor selection and application of data collecting techniques).

Using your Primary Data to Fulfil your RAP Objectives & Research Questions

In drawing up your RAP objectives (which should be very closely related to the Topic Title) your research questions and subsequent approach and data gathering need to be robust enough to demonstrate that you have achieved those objectives.

Remember whatever your topic you will need to ‘create some meaning’ and thus be able to make ‘a judgment’ or come ‘to a conclusion’. However the judgments and any conclusions have to be based on reliable findings, analysis and evaluation. So you MUST have collected sufficient and reliable data to work from. You cannot just conclude that a company’s Human Resource activities or that the introduction of new technology etc. are contributing to or assisting with the attainment of its business and financial objectives WITHOUT PROOF.

Note: Don’t forget if you do primary data gathering you have to include your methodology in the text and your workings in the appendices (The RAP Submission checklist clearly states “If you used a questionnaire as a means of gathering data, have you included your results as well as a sample questionnaire as an Appendix?” (the implication being that if you haven’t you could fail)

What if the implementation/ changed procedure / or new application etc. the organisation has introduced that you investigated has not been successful?

At this point perhaps I need to explain a fundamental of research that students do not always appreciate – although many of the topics such as the ones just cited (Topic 13 and Topic 2) seem to be positively worded they are preceded by the words “An Evaluation” or “Critical review” an evaluation or review is just that – it is an assessment and the result can be negative or positive – it is the reasoned evaluation and the conclusions drawn from it that matter, not the outcome. A parallel to this is in medical research – experiments are done hoping often to find a cure for a particular condition but it would not advance science or be objective to just assume that because research has been undertaken the result is going to be favourable – in reality often it isn’t. The same is true with your RAP it is not the outcome itself that the marker is interested in – as far as they are concerned are they particularly bothered that ABC’s new technology has boosted output or that XYZ’s HR function is selecting the best candidates? I doubt they are concerned about this in the slightest! However what they are interested in, is seeing that you have followed a logical and systematic approach to your research: done sufficient data gathering and have assembled evidence that has enabled you to ‘create some meaning’ and make ‘a judgment’ or come ‘to a conclusion’. They want to see that your work is not just descriptive and that you have not just reported what you found – you thought about it, you understood the requirements (rather than adopted a rote learning approach) and and you have demonstrated that you used the information and have come to a reasoned judgment and logical conclusion!

If you have found that the technology has not done what was hoped then provided you back this up with sufficient evidence then you have achieved your research objective and done “An evaluation of how the introduction of a new technology can assist an organisation in achieving its business objectives” except your evaluation showed in this case, unfortunately, it had not assisted. I once worked for a global organisation that spent a small fortune on employing 2 full time psychologists who administered a battery of psychometric tests to all candidates interviewed for positions. I am sure the HR aim was to get the best people for the posts (and thus contribute to the company’s objectives). Unfortunately they ended up with an extremely high turnover of staff (far higher than I had experienced elsewhere). From this I concluded either the psychologists were recruiting first class candidates and the organisation culture or job descriptions and person specifications were all wrong for those candidates or the tests were totally unreliable and the wrong candidates were being selected via the tests…. so somehow with seemingly the best will in the world, this part of HR policy was not achieving its hoped for objectives! If this company had been my subject company for Topic 13, for part of it I would have used staff turnover rates as a component of my analysis and concluded the policy was failing but also have probably recommended that the company introduce exit interviews and staff surveys to see where it was going wrong (or maybe tried to conduct a staff survey myself as part of my information gathering).

Note: you should be aware that in many organisations you must seek permission before you conduct staff surveys.

See also:

How to tackle Evaluation and Analysis in the RAP Part 1

A Brief Run Through of Evaluation in the Context of the RAP Topics