Do you read books on trains?
The ambiguous question is often hard to spot, but a real nuisance if it’s overlooked.
For example, “Do you read books on trains?” can be interpreted in two completely different ways.
Am I asking you if you read books which are about trains?
Or am I asking you if you read books whilst you are on a train?
The problem is, if a question can be read in different ways by your participants, it will be. And you have no idea which way individuals understood the question and therefore the data collected is quite useless.
A client of mine illustrated this point perfectly recently. He was conducting a survey and wanted to be able to re-contact participants for some follow-up research. It is good practice to gain their permission first, and is indeed addressed in the Market Research Society’s Code of Conduct:-
31. Members must ensure that follow-up contact with a participant is carried out only if the participant’s permission has been obtained at the previous point of data collection.
So, within his survey he included a question:-
Do you mind being contacted in the future regarding anti-social behaviour issues?
Yes
No
Which seems perfectly reasonable.
Or does it?
If they had replied “Yes”, were they saying “yes, it ok for you get back in touch” or were they saying “yes, I do mind if you get back in touch”.
If they had replied “No”, were they saying “no, I don’t mind you coming back to me” or were they saying “no, go forth and don’t trouble me ever again”.
So, when the time came for the follow up, they had no idea who had given their permission for them to do so, and who had not meaning they couldn’t contact anybody.
And I bet you didn’t spot that when you read the question?
A pilot survey is always an excellent approach to uncovering rogue questions. Get an eagle eyed colleague (or two, or three) to go through the survey, completing as different ages, genders, departments, routing options and so on. Invite your client to have a go too. And use the data from the pilot to populate test charts and graphs to check that it us usable and can be illustrated sensibly.
The Survey Initiative team are experts in the field of employee research. If you would like to know more about how we can help your organisation with employee engagement, then call us on +44 (0)1255 850051 or visit our website.