Time to Game Your Survey!
Seemingly not enough of us are making research a fun experience for participants.
How simple is writing a questionnaire? Actually, quite difficult to do well, if the ongoing drop in participant engagement levels are anything to go by.
Research tracking abandonment levels of a 10 minute annual tracker conducted by a company called Researchscape headlines the following:
23% of participants abandoned in 2017
Drop-out levels rose to 53% in 2022.
Wow - over half dropping out. And a huge increase in drop-outs over a 5 year period.
The article highlights the increasing frustration with LOI - 10 minutes, which the authors indicate was the guidance over the recent years, now seems too long. Mobile participation is quoted as a driver in this - as is the level of over- or underpromise: saying a survey will take say 5 minutes and the reality is much longer isn’t popular. TikTok timeframes are likely the unspoken backdrop.
You can read more here - The Ideal Survey Length. Researchscape’s Chief Research Officer - the venerable Jeffrey Henning - suggests that 8 minutes is the new 10 mins in terms of optimising LOI.
There is a discussion on LinkedIn here - Is the Ideal Survey Length 20 mins? touching a bit on aspects other than LOI to improve engagement, reduce satisficing.
But it’s not that heated. As of today, there are 8 comments - which for such shocking data, and coming from somone as authoritative like Jeffrey - is a bit underwhelming.
And it’s not a new topic. I remember listening well over 10 years ago to James Sallows at a London MR conference, who was then working a panel provider from memory. He suggested responsibility for response rates lay with those writing the questionnaires, less with panel providers.
Fast-forward - and things have seemingly gotten worse :(
I’m sure the big guys are on this topic - Jon Puleston for one no doubt has lots to say about how to gamify MR, to get people involved in a topic. Turning questions into quests, for example, by using role-playing scenarios is one approach mentioned in an ESOMAR paper from 2012 (!) which he co-authors with Duncan Rintoul, both of GMI fame then.
Betty Adamou specialises on the topic - check out her website: Research Through Gaming
And there are plenty more, no doubt.
So what’s going on - or down?
Research and the Shopping List Problem
A couple of thoughts.
Firstly, the trend towards DIY is seemingly on-going. Clients writing their own surveys, designing their own questionanires.
How many of these have access to say an in-house “creative” department that can gamify a survey, or even introduce just a few visually entertaining aspects? I’d suggest very few.
Second, in-house researchers have the full exposure to a business’ knowledge need - from all sorts of departments.
Each of which has a specific question about a given topic.
This can easily result in shopping list - lots of detailled questions, that are often asked regardless of the relevance to individual participants.
I recently completed a survey from a bank about sustainaiblity. It went into masses of detail - and took over 10 minutes.
Sustainability may be an important area for a bank - but someone would need to explain to me first exactly how. And there are plenty more pressing issues that I would love to “share” with my bank - eg the difficulty of accessing any sort of unbiased advice. Or their fee structure. None of these was touched on.
Outlook
Looking forward - will the future be different?
The economics of heightened engagement levels would suggest that improvements make sense.
Perhaps ChatGPT or some future development on AI will crack survey design.
But when? It may well be that humans are heavily involved for quite a few years to come. And as MR is thoroughly democraticised, with all sorts of medium and small sized companies engaging in the business of data-based decision making, surveys are likely to remain the plaything of people who don’t have the level of survey design expertise as say Mr. Puleston.
My take: concentrating on the essentials in any survey design is a great way forward. Most people will tell you what’s really important to them quickly, if you give them the chance.
Getting stakeholders to buy into that may be tricky, but valuable - finding ways of filtering sensitively so that a population can be divided into segments on a “matters to me a lot” or “matters to me a little”, and then treated accordingly.
In a nutshell, keeping research engaging is key, using the principles of gamification make sense - but even more importantly, we need to ask less questions. This can be truly valuable and revealing if the approach is right, as my qual. colleagues know only too well.
Curious, as ever, as to others’ views.
(Photo: Photo by Priscilla Du Preez on Unsplash)