Transcription
ESOMAR 28Ipsos answers to ESOMAR 28 questionsto help online research buyers1. Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information and.may not be disclosed or reproduced without the prior written consent of Ipsos
203About this document04Company profile06Sample sources & recruitment11Sampling & projects management22Data quality & validation30Policies & compliance
ABOUT THIS DOCUMENTThe primary aim of the ESOMAR 28 Questions is to increase transparency and raiseawareness of the key issues for researchers to consider when deciding whether an onlinesampling approach is fit for their purpose. Put another way, the aim is to help researchersto ensure that what they receive meets their expectations. The questions are also designedto introduce consistent terminology for providers to state how they maintain quality, toenable buyers to compare the services of different sample suppliers. Notes on the contextof the questions explain why the questions should be asked and which issues researchersshould expect to be covered in the answer.These new questions replace ESOMAR’s “26 Questions to help Research Buyers of OnlineSamples”. ESOMAR has updated the text to recognize the ongoing development oftechniques. While some of the questions remain constant, new questions have been addedto incorporate new techniques and new technology in this area. In particular, this revisionrecognizes the broad trend within the industry to build online samples from multiplesources rather than relying on a single panel.With over 4,5 million panelists in 50 countries, the Ipsos Online Panel has one of the mostrobust coverage in the world, with panels in all major markets in Europe, North America(NA), Latin America (LatAm), Asia-Pacific (APAC) and Middle-East (ME). Ipsos has thecapability to conduct online research anywhere in the world where Internet penetrationand usage allow. Today, panels are the largest sample source for online studies. They arepowerful and tactical research tools. Thanks to our global organization, our clients benefitfrom the advantage of a worldwide network, combined with high and consistent qualityacross countries.It should be noted that these 28 Questions focus on the questions that need to be askedby those buying online samples. If the sample provider is also hosting the data collectionyou will need to ask additional questions to ensure that your project is carried out in a waythat satisfies your quality requirements.The 28 Questions complement ESOMAR’s Guideline to Online Research which was revisedin to add updated legal and ethical guidance and new sections on privacy notices,cookies, downloadable technology, and interactive mobile.3
COMPANY PROFILE44
1. What experience does your company have inproviding online samples for market research?Context: This answer might help you to form an opinion about the relevant experience ofthe sample provider. How long has the sample provider been providing this service anddo they have for example a market research, direct marketing or more technologicalbackground? Are the samples solely provided for third party research, or does thecompany also conduct proprietary work using their panels?Ipsos’ commitment to panel research quality is a tradition extending back to our mailpanel heritage in the 1970s and ‘80s. This formed the foundation of our global onlinepanels and our research-on-research program, which has helped us develop bestpractices for achieving and maintaining high-quality data and respondent engagement.There are only a few other companies with this heritage and extensive expertise inmarket research and panels.Ipsos Interactive Services has panels in over 70 regions in North America, Europe, LatinAmerica, Asia-Pacific, and the Middle East. Its divisions adhere to the same onlineresearch industry principles and panel management guidelines, though some differencesin regional approaches exist. Rules and privacy policies may vary to reflect the needs inthe respective regions.Ipsos began conducting online surveys in 1999 and has been running online panels formore than 15 years. We have conducted almost every type of research online. Studytypes fielded include (but are not limited to) concept, package and product testing,website evaluations, public opinion, sensitive topics, voter exit polling and ad testing.5
SAMPLE SOURCES& RECRUITMENT66
2. Please describe and explain the type(s) ofonline sample sources from which you getrespondents. Are these databases? Activelymanaged research panels? Direct marketing lists?Social networks? Web intercept (also known asriver) samples?Context: The description of the types of sources a provider uses for delivering an onlinesample will provide insight into the quality of the sample.Ipsos Interactive Services is able to guarantee excellent market research results thanksto our experience in panel management within the Ipsos Group, both offline and online,and because Ipsos was one of the first companies to embrace the switch from traditionalto online data collection methodologies. Our panels are not just lists or databases ofindividuals, but actively managed research Access Panels: Individuals who have volunteered to take part in market research surveys Created and managed for long-term use and access Extensively profiled to efficiently target respondentsOur experience has shown that the sample source plays a vital role in a company’s abilityto make accurate business decisions based on the facts. Ipsos’ online panels arecontinuously refreshed using a variety of sources and methods. We firmly believe thatdata quality begins with recruitment. The vast majority of our panelists are referred tous through various online suppliers. Based on our many years of experience, we knowthat multi-source recruitment is the best option for maintaining a representative base ofrespondents. We only use high quality recruitment sources to entice people who areeager to take surveys.We strategically focus on developing processes that reflect the newest internet practicesas may currently be found through social networks. Email lists, banners, website andtext ads, co-registration, and search engine marketing are also used. When evaluatingthe priority of sources, the recruitment process takes the following into consideration: Panel parameters: ex. response rates, attrition, data quality and demographics Census representivity: Though we consider it in our recruitment strategies, we aimto recruit a selection of panelists who meet business requirements (including censusrepresentative samples for frequently used groups, ex. Females aged 18 49) High demand targets: ex. people with certain ailments or from certain professions.7
professions.3. If you provide samples from more than onesource: How are the different sample sourcesblended together to ensure validity? How can thisbe replicated over time to provide reliability? Howdo you deal with the possibility of duplication ofrespondents across sources?Context: The variation in data coming from different sources has been well documented.Overlap between different panel providers can be significant in some cases anddeduplication removes this source of error, and frustration for respondents.When necessary, Ipsos will utilize multiple sample sources to support client needs. Asample blend consists of specific sources in a fixed proportion used to meet therequirements of a study. We follow a best practice of managing quotas by supplier toensure a consistent sample frame from each source in the blend. To ensure replicabilityover time it is our standard to recommend using 2 or more sources for all trackingstudies. Consistent and controlled blending across multiple sample sources providesaccess to a larger pool of quality sample and provides for reliability across studies / waves.We remove the possibility of duplication across sources through the use of Imperium’sRelevantID service, which prevents duplicate respondents from entering live surveysbased on digital fingerprinting criteria. The RelevantID check is applied to all surveysconducted by Ipsos, including those where mixed panel or non-panel sources are used forthe same survey.8
4. Are your sample source(s) used solely formarket research? If not, what other purposes arethey used for?Context: Combining respondents from sources set up primarily for different purposes (likedirect marketing for example) may cause undesirable survey effects.Ipsos panels are used solely for market research purposes.5. How do you source groups that may be hard toreach on the internet?Context: Ensuring the inclusion of hard-to-reach groups on the internet (like ethnicminority groups, young people, seniors etc.) may increase population coverage andimprove the quality of the sample provided.We constantly carry out targeted recruitment through: Various “wide net” methodologies (ex. email campaigns, affiliate networks, banner ads,text ads, search engine, co-registration, offline-to-online, specialized websites) Customized incentives and materials for recruiting special targets such as mothers ofbabies, age group 55 , etc.Moreover, we are continuously testing new recruitment sources and methods (specializedwebsites, social networks, etc.).6. If, on a particular project, you need tosupplement your sample(s) with sample(s) fromother providers, how do you select those partners?Is it your policy to notify a client in advance whenusing a third party provider?Context: Many providers work with third parties. This means that the quality of thesample is also dependent on the quality of sample providers that the buyer did not select.Transparency is essential in this situation. Overlap between different providers can besignificant in some cases and de-duplication removes this source of error, and frustration9
for respondents. Providers who observe process standards like the ISO standards arerequired to give you this information.Due to the dynamic nature of the online market research business, at times we must relyon third-party online sample providers to complete client-sponsored market researchstudies. We have a tightly controlled supplier vetting process through which we identifycompanies who adhere to quality and respondent management process which closelymirror our own. Companies selected as preferred partners adhere to stringent guidelineson each project commissioned. Additionally, we have a Supplier Quality/Satisfactionprogram through which we capture qualitative & quantitative measures on supplierperformance which are then reviewed monthly by management.10
SAMPLING & PROJECTSMANAGEMENT11
7. What steps do you take to achieve arepresentative sample of the target population?Context: The sampling processes (ex. how individuals are selected or allocated from thesample sources?) used are the main factor in sample provision. A systematic approachbased on market research fundamentals may increase sample quality.Sampling begins by developing targets to match official census statistics or profiles asrequested by the client for any given country. For certain target populations, such asmobile phone owners, pet owners, etc., we have the ability to generate representativequotas based on a sub-sample of the panel, and then use these quotas to obtain abalanced sample of the target population.Sampling is carried out via a proprietary sampling application that allows us to constructcomplex samples based on the target and screening requirements. The software selectspotential respondents that balance according to the targets (ex. selected to berepresentative of the general population, internet population or other specificdemographic profile required).The software uses an interactive selection algorithm that balances one variable at a time inorder of priority, as follows: The first step is to extract all active and available panelists that meet the screeningcriteria (ex. demographic, geographic). The sample pool is randomly sorted. The algorithm then examines the first (primary) variable and selects the number ofpanelists who satisfy each target. (Sometimes, there may not be enough availablesample to fill all cells and since some variables are more important than others, lowerpriority variables may not balance precisely). Finally, the sample may be distributed and balanced among more than one cell so thatdifferent treatments or surveys may be fielded in equal balanced groups or cells.Exclusion procedures: panelists will be eliminated from participating in surveys forspecific periods of time. Ipsos applies its own exclusion rules which take into account thetype of study, the category researched, the number of surveys respondents have alreadyparticipated in, etc. These rules are based on our panel management expertise and areaimed at eliminating the bias resulting from overusing the same respondents, whilemaintaining panelist engagement.Deployment: The sample can be deployed in batches using any criteria requested by theclient. This is controlled by an application used in the management of the sample mailouts. The mail-out tool allows automatic mailing at a predefined time/day and bybatches.12
We can also control the number of completes for any specific cell by setting desired endquotas on any specified criteria (ex. gender, age ). Once a target quota is achieved, thesurvey will close for respondents within that target.8. Do you employ a survey router?Context: A survey router is a software system that allocates willing respondents to surveysfor which they are likely to qualify. Respondents will have been directed to the router fordifferent reasons, perhaps after not qualifying for another survey in which they had beendirectly invited to participate, or maybe as a result of a general invitation from the routeritself. There is no consensus at present about whether and how the use of a router affectsthe responses that individuals give to survey questions.