Global guideline to safeguard integrity of $10bn online res market26th Mar 15
The Global Research Business Network (of which AMSRS is a member) and ESOMAR have collaborated in publishing guidance to ensure the integrity of sample quality used in online research.
A growing global market worth US$9.7bn (more than phone and face-to-face combined), online sampling now accounts for 28 per cent of global market and social research revenues.
According to professional standards consultant Reg Baker its rapid growth has been accompanied by concerns about the integrity of the resulting research data and presents ‘real challenges for brands and marketers to assess the quality of their samples’.
The Guideline for Online Sample Quality (GOSQ) provides best practice guidance to help users and providers understand how evolving approaches to online sampling can impact and potentially compromise data quality.
It emphasises the need for a transparent sampling process, knowledge of how the sample was selected, and specifically to ensure the same individual only answers a particular survey once.
Issues include the number of professional research participants who enter multiple surveys to secure incentives, inattentive or untruthful respondents, unrepresentative target groups, and the potential for duplicate participants as research providers broaden their sources to expand sample sizes.
Between 20 and 30 percent of research respondents now respond using a smartphone or mobile device – they are typically younger, male, and more ethnically diverse than in conventional sampling. The limitations of mobile devices can also skew completion rates. The guideline highlights
the need for transparency when reporting data in this area.
Newer techniques such as ‘river sampling’ – recruiting samples from real-time adverts or offers – can make it harder to validate respondent identities and their relevance to the target population.
The trend towards recruiting respondents from schemes such as ‘frequent flyer’ programs or from particular websites opens up the potential for duplicate participants. Blending together samples
from several panel sources to create greater balance can also lead to individuals answering surveys more than once.
GRBN executive director Andrew Cannon said future planned collaborations with ESOMAR include working with industry experts to publish guidelines in other rapidly evolving fields such as social
media and mobile research.
- Ensuring the respondent falls within the description of the research sample.
- Survey fraud prevention to ensure that the same person cannot receive more incentives by completing a survey more than once.
- Survey engagement to ensure that the respondent is paying sufficient attention to the questionnaire and understands the questions being asked.
- Category exclusions to ensure that the sample does not include respondents who might bias the results.
- Transparency of sampling to build confidence.