How to Conduct Online Qualitative Research to Test Ads

by Paul Rubenstein, Ph. D. (President, Accelerant Research)

For decades, qualitative research has been used to inform advertising and communication strategies and to test ads before they are made public.  Steps in the development process usually have organizations (and their agencies) moving from creating advertising concepts, to designing story boards and animatics, to assembling final executions.  All of this, often times, is fueled by the insights gathered at each point using traditional focus groups.

And it is the focus group that has been the predominant method used to achieve ad testing related study objectives.  Data collected in these studies are used to inform strategies and tactics in terms of what to say and how to say it.  That is because consumers’ opinions about the subject matter and specific stimuli to which they are asked to opine is used as the foundation for advertising and communications.

Particularly when brainstorming is needed among consumers, conditions that foster creativity can be facilitated by a group setting.  Being able to hear others’ ideas helps individuals come up with ones of their own.  Furthermore, role playing and other projective techniques represent tried and true exercises that have been used successfully in focus groups conducted for ad testing purposes for many years.  Simply, a moderator is not worth his salt if he has never used some of these techniques, at least sometimes, and certainly for ad development purposes.

However, there are several highly effective projective techniques and other kinds of exercises that are highly problematic to execute in an in-person setting such as a focus group facility. Whether it is perception mapping or sorting, story-telling projectives, image tracking or text tracking, or dial-testing exercises that are needed to be deployed, using printed materials, markers, scrap paper, glues, pins, dials, or spit and tissue paper, successfully managing these kinds of research techniques are among the greatest challenges to a moderator.

In contrast, shifting data collection methods from face-to-face (F2F) to an (asynchronous) online qualitative research platform that has these functions will not only produce more data and cost less than focus groups, but will enable these sorts of advanced research techniques to be utilized more easily and the data gathered will be analyzed with greater precision.

To begin, let's understand the entire research agenda involved at various steps in the development and execution of advertising campaigns.  The figure below may serve to do this in one graphic.

As stated earlier, the entire end-to-end process involved in advertising development can be boiled down to three (3) simple questions:

1. What should the ad say?
2. How should it be said?
3. How well did the ad work?

These fundamental questions can be mapped to specific types of research studies, namely, consideration, articulation and execution, and evaluation studies, respectively.  Beginning with consideration research, the main objective is to understand the equity of a given brand within the context of its competitive landscape.  As such, the researcher must uncover and reveal how consumers conceptualize the industry in question and which key characteristics drive their consideration of one brand over others.

Consideration Research

A useful technique for consideration research is perception mapping, which can be executed in both quantitative as well as qualitative research.  In quantitative research, a set of multivariate techniques are brought to bear, including discriminant function analysis, correspondence analysis, or multi-dimensional scaling.  The resulting perception map that is formed from the pattern in the quantitative data shows how target consumers view the similarities and differences between brands and which brand characteristics define each best.

But a highly useful, first phase of consideration research may warrant a qualitative study that is used to inform the subsequent quantitative phase.  In this case, a perception mapping exercise, similar to what has been used in focus groups for so many years, may be improved in its execution if done online.  Through simple and fun "drag-and-drop" motions that participants are instructed to perform, a resulting perception map may be derived accurately and easily, as shown below in a mock example:

As the map shows, the position of each logo is automatically determined by the programming of the platform which is set to place each one at the average X and Y bi-variate coordinates calculated from the pool of participants in the study.  To do so accurately using F2F methods is almost impossible.  Furthermore, using an online qualitative research platform for this purpose enables the moderator to filter the data with a button push or two in order to test any study objectives that warrant subgroup analyses.

For that matter, other projectives that require drag-and-drop may also be used.  These include various sorting exercises in which the number of or labels for the categories may or may not be provided in advance, such as shown below:

 

And yet another projective technique that has been utilized for decades in focus groups is what is known as “Story-Telling.”  This technique uses a set of images that evoke various positive and negative emotions in people.  These images are shown to participants who are asked to choose, from the set, ones that they then put in some order and use to tell a story about the subject matter.  This exercise is most easily facilitated online through drag and drop functions built into the platform and yield individual responses that are captured along with (text-based) stories.  Moreover, these stories integrating images and text can be readily copied and pasted into the moderator’s report with a few button pushes, as opposed to the tallying (by hand) of each story, each image, and each participant’s ID number as would be the case in a traditional focus group setting.

Articulation Research

In Articulation research, the main objective is to inform “how to say it.”  This, too, can entail integrated phases of both qualitative and quantitative studies, one informing the other and each designed to do just that.  Regardless of the type of study, it will include gathering the reactions from target consumers to a distinct advertising/communication stimulus, e.g., print ad, advertising concept description.

For these types of qualitative studies, moderating discussion among participants to generate their ideas for ads is usually a part of the process.  In addition, participants also are shown various articulations that have been developed by the organization and/or its ad agency.  These stimuli then become the grist for the mill to sharpen, clarify, improve, or even eliminate by using functions such as image tracking or text tracking.  As can be seen by the example below, participants can isolate portions of the stimuli according to the moderator's needs, such as "draw a red circle around the portions you like." 

Likewise, in the case of text tracking shown below, the instructions to the participant may be to "highlight the specific words you find confusing."

And if the stimulus is a video as opposed to a static image, such as a TV commercial, dial-testing is arguably the best and most sophisticated technique to evaluate this kind of stimulus.  It directly shows at what point during the span of time of the commercial (e.g., 30 seconds, 60 seconds) participants liked or disliked the ad and is usually utilized after a finished ad has been developed and is about to air.

All of these articulation testing exercises have been used in focus groups for a long time, for at least the past 20 years.  I remember many years ago having to carry the heavy, sometimes reliable dial testing equipment to the special facility with auditorium seating to run dial-testing focus groups.  These groups were very expensive for the client and nerve racking to the moderator as dials, wires and their connections would, sometimes, be faulty.

But nowadays, thanks to online qualitative research platforms, all these functions are facilitated very easily, and far less expensively.  Aggregating the data, creating output, and including it in final report deliverables is handled by some clicks on the screen. 

This is a great time in the history of the market research industry to be in this business.  The confluence of broadband connectivity, digital technology, and the comfort level and skill of consumers in communicating so easily and so well within computer-mediated environments has made the need to conduct F2F qualitative research become increasingly threadbare.  Truly, the net effect of these factors in combination has opened up a whole new world of possibilities in social research in general and market research in particular.   

Considering that focus groups have been done the same way for over four decades, qualitative research was very much in need of some fresh approaches.  Conversely, quantitative research has undergone radical shifts and improvements as a function of advances in technology over this same period and that have improved data quality and reduced cost and time requirements.  Indeed, it is a rare occasion when quantitative surveys need to be administered F2F, e.g., mall intercepts.

Indeed, qualitative researchers will be ever more hard pressed to rationalize to clients their choice of F2F and why they are not shifting to an online method for studies like these.  Indeed, better, cheaper, and faster is a compelling argument that forces client-side research managers to embrace online methods as they, too, will need to rationalize their choices to the clients they serve within their organizations.

For more information about our services, please visit us at www.accelerantresearch.com or email us at info@accelerantresearch.com or call us at 704.206.8500.