Bounty, Bias, and Employee Surveys: Lessons from a Chocolate Poll

Just before Christmas, one of our team brought in a box of Celebrations chocolates. As a research company, it was almost inevitable that this would turn into an impromptu poll and data analysis exercise. However, some surprises in the results (namely Bounty being the second most popular chocolate behind Maltesers) raised conversations about two issues that can seriously undermine the value of employee engagement surveys: leading questions and confirmation bias. 

Celebration ranking results

Leading questions

Leading questions are defined as questions that may cause a respondent to answer in a biased or particular way, often a way that aligns with the goals of the asker. Because leading questions are often unintentional, it is especially important to know how to identify them (Webb, 2017). 

By describing Bounty as an outlier and as a chocolate that should not be eaten as a first choice, the prompt inviting participants to our chocolate poll could be considered a leading question. This framing unintentionally suggested that Bounty was an inferior choice among the Celebrations. 

Unfortunately, leading questions are not limited to chocolate ranking polls; they can have several implications for the validity of employee engagement surveys. Perhaps the most significant effect is that employees who disagree with an implied position may feel that their true opinion is unwelcome, consequently causing their response to gravitate towards what they perceive to be the desired answer. 

At the same time, leading questions can motivate respondents who strongly support a particular option to participate in order to defend it – which may have been the case in our chocolate ranking survey. How else would Bounty rank so highly 😉? 

Together, these effects can significantly distort survey results. In extreme cases, this may go beyond minor bias and may dramatically skew the overall sentiment captured by the survey. 

Furthermore, when participants develop a perception that questions are worded to elicit a certain response, they may view the survey as performative rather than genuine. This may reduce trust in the survey process and weaken employees’ belief that their feedback will lead to meaningful action. 

However, despite the problems discussed, leading questions are easy to address. To mitigate them, it is important to ensure that questions are worded in neutral language, avoid assumptions, and offer balanced response scales with an equal number of agree and disagree options. 

Confirmation bias

In addition to leading questions, the recent Chocolate poll prompted discussion about issues that can arise when interpreting the results of a survey. One such issue is confirmation bias, which refers to a human tendency to seek out and interpret information in ways that align with our pre-existing belief (Casad, 2026).  

As someone who does not like Bounty – and who placed them at the bottom of my ranking (only ahead of Snickers) – seeing them come second was surprising, and something I attributed to a flaw in the poll. However, after taking a step back and reviewing the results from an objective perspective, it became clear that I had fallen victim to my own confirmation bias: I had expected Bounty to rank poorly simply because they are not my preference. 

Just as with leading questions, confirmation bias is not limited to chocolate-ranking polls and can have significant implications for the usefulness of employee engagement surveys. When reviewing survey results, confirmation bias can lead to overemphasising findings that align with prior expectations, while downplaying unexpected results. Negative comments or low scores may be dismissed as outliers or the opinions of a few disgruntled individuals, rather than being recognised as potential indicators of wider issues. 

When actions taken in response to survey results are influenced by confirmation bias, employees may lose confidence that their feedback will lead to meaningful change, potentially creating apathy towards future surveys. However, as with leading questions, confirmation bias is relatively easy to overcome. Ensuring an open-minded approach to survey results – consciously setting aside existing beliefs (or tastes, in the case of the chocolate poll) – allows for more accurate interpretation of findings and more effective action. 

One sure-fire way of avoiding leading questions and confirmation bias is to involve a third party in the survey process. By bringing in people who are experienced in question design and are not emotionally invested in the results – or have preconconceptions, organisations are able to gain a fresh perspective that challenges assumptions, ensures questions are neutrally framed, and uncovers insights that might otherwise be overlooked. If you would like support with your employee survey, to ensure your data is accurate, balanced and genuinely useful, contact us – we’ll be happy to help!

Sources 

Casad, B.J. (2026) Confirmation bias | Definition, Examples, Psychology, & Facts. Encyclopaedia Britannica. Available at: https://www.britannica.com/science/confirmation-bias 

Webb, N. (2017) ‘Survey: Leading Questions’, The SAGE Encyclopedia of Communication Research Methods. Available at: https://methods.sagepub.com/ency/edvol/the-sage-encyclopedia-of-communication-research-methods/chpt/survey-leading-questions