Market research in many ways is the industry ‘comfort blanket’ that underpins marketing plans and assumptions. If you can say “It’s not just me saying this – our consumers agree and here is the evidence”, then it’s that much harder for your plans to be argued with. And yet – favourable research results are no guarantee of success. Let’s dig into some of the reasons behind this timeless marketing conundrum.
Market Research – The most abused marketing discipline?
There is an excellent episode of the great 1980s BBC comedy series “Yes Minister” which features a demonstration of exactly why market research into popular opinion is eminently unreliable.
In a beautifully written scene, the junior civil servant tells his boss about a recent opinion poll showing that voters are in favour of a contentious new government measure (in this case the restoration of National Service). The senior man, in charge of the department, is not pleased to hear this as he is against the measure – and instructs his colleague to run another survey to prove the opposite result.
This completely throws him – he can’t see how a piece of research can be commissioned to contradict the findings of another. Surely, the findings of the first one should be definitive. The point is then beautifully made that research participants can be ‘led’, like a court case witness, to provide exactly opposing views on the same subject by some clever leading questioning. If you want to know how this is done, watch the clip below:
I’m not saying all research is so shamelessly manipulative as this. But it is a fact that a lot of market research is commissioned with the objective of proving that an assumption is correct – and the research structure, therefore, tends to mitigate against any other answer other than the one it sets out to achieve.
This is surely not what market research was originally designed to do. And it’s not what most people assume any individual piece of research is designed to do. So how has it come about? And what can be done to redress the balance so that market research results can be better relied on when they influence important marketing decisions?
How did we get here?
Part of the issue is cultural. When business decisions are being made, it’s comforting to have a view from the customer as to the impact of what is being decided. A cynic might say that this is a way of deflecting the blame should the decision turn out to be the wrong one, but the rational observer can see the sense in this.
However, it does result in abuse of the system. People who have invested time, energy and reputation into developing a new initiative are understandably keen to get the idea to run and are not above loading the market research strategy in their favour. I guess this has always happened – the famous Henry Ford dismissal of market research is many decades old (“If I’d asked the customer what he wanted, he’d have said a faster horse…”). But it does seem to be rifer in business today.
We can see a few reasons for this. Less scrupulous market research companies willing to take on ‘pre-judged’ research projects might be one cause, but there are deeper trends that marketers need to wary of.
One is the increased involvement of procurement teams in marketing budgets. Their impact is felt everywhere and the pressure is usually to drive down costs. Market research suppliers, therefore, have to find cheaper ways to get things done, and this has an inevitable impact on quality. Smaller samples, less focus on recruitment, less time spent on the development of questions and results interpretation – we can see evidence of all this going on as cost pressure is applied.
The arrival of standing internet panels has given rise to a second recent trend – the semi-professional market research panelist, seeking out enrolment as a paid member of multiple panels. They know what the questions are seeking to find out and are quite capable of playing their own games in replying.
A third development, and probably the most devaluing one of all for market research, has been the arrival of online instant research tools such as Survey Monkey. There is a strong clue in the name – any idiot, it implies, can run a market research project. And many idiots have. The findings they come up with often have little grounding in science or statistics and are often used to generate click-bait headlines like the one above this article. (I plead innocence here – no research of any kind was used to come up with this one. I’m sure I could have used Survey Monkey to generate the evidence, but I decided not to bother so it’s purely anecdotal.)
Here’s a much older problem – marketing teams can also be their own worst enemies. How many times have we seen the senior team gather to view the interim results of a research project and take a snap decision based on what they hear there, without waiting for the full results and the nuances that come with it?
The problem with Self-fulfilling research results
The impact of this lack of best-practice discipline can be far-reaching on the business.
Here is an excerpt from an April 2021 article by Matthew Syed of the London Times regarding recent research run by the English Cricket Board on cricket terminology. “According to a report, the term “wicket” has been deemed too onerous, so the ECB is considering replacing it with “out” for the new format known as the Hundred. So, instead of a team being 93 runs for three wickets from ten overs, it will be 93 runs from 60 balls for three outs. The goal is that the game should be based on three concepts: “outs”, “runs” and “balls”.
Inevitably, this notion has emerged because the ECB is keen to bring “blue-sky thinking” to the Hundred. It has conducted market research and worked with focus groups.”
The ECB set out to find ways of making the game more accessible for a younger under-25 audience and fell into the trap of presupposing that the specialist language of the game was a barrier to understanding it. They have failed to dig a little deeper and see that maybe the language is part of the appeal of the game and that, actually, gaining insider knowledge such as the meaning of specialist terms is a key to becoming a true fan.
So instead, they are choosing to adopt terms and words from a different sport that they see as having a wider appeal to a younger audience – in this case, baseball from the USA. A broader interpretation of the results might reveal that the USA has a population of 332 million, only 21% of whom are under the age of 25 and not all of whom necessarily like baseball. Whereas, the world cricket epicentre in India has a population of 1.38 billion, 47% under the age of 25, are already cricket crazy and apparently all fans of the specialist language of the game.
There are innumerable other examples of this kind of limited thinking. Important, business-changing decisions are made off the back of research that has been misinterpreted or constructed specifically to prove a pre-conceived theory. Results are further undermined by an inadequate briefing of the research, and by short cuts being made due to budget restrictions or poor practice by the researchers.
It’s a dangerous place to be for business and it needs urgent attention.
What are the lessons for marketing teams?
To go back to Matthew Syed of the Times – “Sadly, (the ECB) didn’t get the memo during these “brainstorming interfaces” that market research needs sensible interpretation.
He’s right. What we all need to remember is to read all the research findings and to make sure we understand the methodology behind the report. How many people were questioned? Who are they? How were they sourced and recruited? Were they paid, and how much? Above all, what is the full context of the research findings including the questions that preceded the major ones?
Interpretation is all. And far too often, marketers are their own worst enemy for wanting instant results with minimum attention to the briefing and the debriefing process. If we do nothing else to improve the situation, let’s at least make an end to the eternal request to “Just give me the topline!”
*This figure comes from ‘market research’ carried out by the author. It has little basis in quantitative data and is designed to catch your attention and nothing else – please treat with caution. You can see how all this works…
TrinityP3 has solutions that address the challenges marketers are facing from strategy development, budget setting, performance metrics, business growth alignment, marketing structures and stakeholder engagement and alignment. Learn more here
