EmergeSmarter Blog

By Shaili Bhatt, Senior Analyst

Projective questions from in-person interviewing flow smoothly into online qualitative activities! Nancy Hardwick gave a wonderful presentation on projectives at the 2011 QRCA Conference in Las Vegas, and here we explore how these activities work well online.

Most, if not all, of my favorite projective activities in new qualitative research are derived from traditional qualitative research methods.

Online discussions or communities feature an extended interviewing phase (multiple days, weeks or months beyond traditional 2-hour focus groups), which essentially provides researchers with a welcome abundance of time to harvest and probe an always-impressive incoming flow of information. How can researchers effectively utilize this time to engage online participants and immerse them in the topic at hand?

For the time being, quick “top-of-mind” free association exercises are just as important as creative projective questions, requiring online participants to reflect for a few moments (or days!) to capture a particular feeling or experience in a thoughtful post or activity page.

Written exercises like storytelling and other creative activities like collages, when used at opportune times, can be the key ingredients to insightful and interesting new qualitative research. No stimuli are required, and natural dialogue helps to tie it all together.Nancy Hardwick of Hardwick Research presented, “Projectives in Practice,” a detailed compilation of projective techniques at the 2011 QRCA Annual Conference, which was held at the luxurious Venetian & Palazzo Resort in Las Vegas.

Hardwick encouraged the audience to interact and build upon the listed activities during her in-depth, power-packed presentation. While the focus of the presentation highlighted “what works” with in-person interviewing, the ideas and energy in the room quickly catapulted this to a presentation that refreshed my perspective and sparked the most NewQual inspiration in my notebook.

(As you may know, projective techniques are subjective questions that researchers use to elicit the underlying emotions or subconscious drivers that influence choice, as an alternative or complement to asking direct questions. Many of these techniques originally stem from projective personality tests in Psychology, which were designed for people to respond to fairly nebulous, ever-inconclusive stimuli, presumably uncovering hidden emotions and internal conflicts in the process.)

Throughout the presentation, Hardwick included a steady stream of projective techniques, resulting in a compilation of audience favorites in several important categories:

• Written Exercises
• Photo/Drawing Exercises
• Sorting Exercises

The variety of projectives serve as a reminder for how many of these time-tested exercises can be incorporated into online research.

Written exercises transition smoothly into the online world.

A few of my favorite activities are as follows (sans embellishments):

• Famous Owners—Pick a favorite/popular celebrity, and describe 1-2 thoughts about who they are. Then, ask what this celebrity’s version of (insert client’s product/service) would be like? What would it look like? How it would perform, and why? Note: This is a great technique for exploring an existing client’s products or services as well as innovation and co-creation.
• Storytelling: Describe a specific experience from the last time that you…(insert scenario). What Did You Think? Say? Feel?
• Tribute/Eulogy—Pretend that (insert product/service) no longer exists. What did it accomplish? What will you miss most about it? Describe all that you feel and want to say about the product, even if you are viewing it in a new light. Note: variations on the theme, such as a “Lifetime Achievement Award” can be more attractive for certain products/services, particularly category leaders.
• Picture Your World—Pick a picture/color that represents how you feel about (insert product/service). How does this picture/color represent how they feel, and why?

There’s no need to reinvent the wheel.

Very often, writing a discussion guide for online research is about applying techniques that are already available and crafting projective activities to capture the level of insights you can hypothesize…and then some!Hardwick advises to begin with a written exercise “early on” in the discussion, which also works well with new qualitative –even if it’s typed; this early activity serves to challenge the participants to think independently and also allows them to “own” their ideas and suggestions. A well-executed written activity that provokes thoughtful posts is also a great way to encourage insightful group discussions.Online discussion guides written like quantitative-type surveys, with numerous objective questions, can certainly be cringe-worthy. Please do not be afraid to ask creative in-depth questions: the online medium suits that well!

Leave room for the unexpected when writing questions and avoid mechanical dialogue. Don’t be overly repetitious.

When using projectives, encourage honesty and spontaneity while asking for details. I like to throw in a few natural quips at the end of questions, for example, “It could be anything!” (As in, “What’s your favorite part about ______? It could be anything!”) It is interesting to note how such a brief invitation can lead to a wider variety of posts. Treat it like a conversation, because it is!

By Robert Relihan, Senior Vice President

If you have conducted focus groups, or even observed them, you have probably noticed that the energy level can vary over the course of a day.  I have always taken this as the normal course of events. But, it turns out there is a rather interesting explanation for this ebb and flow — decision fatigue.  A recent article by John Tierney in the New York Times, “Do You Suffer From Decision Fatigue?,” describes the research surrounding this phenomenon in great detail.

I had always thought people simply get tired during the course of the day, but Tierney demonstrates that the mere act of making decisions wears us down, making us less able make additional choices.  He points to a study in which individuals who simply review material were better able to make choices about it at the end than those who were forced to repeat incremental choices. 

He also notes that those who receive food make better decisions.  And, it is not the psychological reward of the food that works.  Tasteless, but sugared grub will have more effect than a sugar-free sundae.

What is the impact of decision fatigued?  One makes poor decisions, of course.  But, what is a poor decision?  The fatigued person defaults to the familiar, the usual, the expected.  In other words, he or she avoids making the hard choices — just those choices we would like research participants to make.

So, what is the lesson here for researchers?  Well, I am going to take the cans of Pepsi and cookies on the table much more seriously than I have in the past.  I might even ban water and diet beverages.  On a more fundamental level, it has caused me to re-think the number and nature of exercises I ask people to perform in a focus group.  Let them create a collage at home rather than force them to select pictures for a collage during the interview.  Might there be implications for questionnaire design?

Concerns about decision fatigue also make on-line qualitative and MROCs look all the more promising.  Participants can respond to our tasks at the times they are freshest and best able to evaluate whatever it is we wish them to judge.

There is another troubling issue Tierney raises.  “Decision fatigue is a major — and hitherto ignored — factor in trapping people in poverty. Because their financial situation forces them to make so many trade-offs, they have less willpower to devote to school, work and other activities that might get them into the middle class….  A trip to the supermarket induces more decision fatigue in the poor than in the rich — because each purchase requires more mental trade-offs.” At first this seems like an important, but a political concern.  Yet, it has implications for those who wish to understand the behavior of consumers.

Dan Peck’ Atlantic article, “Can the Middle Class be Saved?” described what he calls the “hollowing out” of the American middle class.  His economic arguments are familiar. Over the past several decades, a larger proportion of the nation’s wealth has accrued to a smaller proportion of individuals at the top.  But, in the process, the middle class has fractured.  Now, the non-professional middle class are more likely to resemble in behavior and attitudes the “high school drop-out poor” than the “college educated members” of the middle class.

The sub-text — many more of those whom we survey or wish to have in our MROCs and focus groups are likely to be in a semi-permanent state of decision fatigue.  Our methods and approaches will need to be much more sensitive to this fact.

By Robert Relihan, Senior Vice President

I am staring at a series of collages I had participants create for a project I just completed.  I have always loved this sort of creative exercise.  It helps (forces?) people to stretch their right brains.  They make more connections and associations when they create the collages and discuss them than they would in almost any other exercise I use in a focus group.  Also, I discover symbols and metaphors reflected on the collages and in the responses of their creators than with almost any other analytic approach.

I was surprised to read an article in the New York Times recently that seemed to suggest something novel in the technique.  I have been using it for years.  One of my approaches is to have participants create collages out of whatever pictures and objects they have around the house.

As I looked as those collages, I realized they said much about the ways a group of young men and women felt about one of their favorite activities, which shall remain nameless.  I could see changes that had taken place in the way creativity is expressed by people who are not specifically trained to be creative.

• Fifteen years ago a typical collage would contain pictures from a range of magazines.  And, they would also be taken from both the advertising and the articles.  Now, it seems that people’s image frame of reference is much more restricted. It is not unusual to see the same pictures repeated in several collages.  It is evidence that magazines are much less a part of everyone’s daily life.  When I conduct ethnogaphies, many houses I visit have no evidence of magazines.
  
  It was commonplace for collages created by men to have pictures from the sports page of the local newspaper or from Sports Illustrated.  No more.  All sports news is on-line.  And, as more and more people use smart phones, they are getting their sports news in such a way they can’t even print a picture.
  
  And, the images that do appear on the collages are almost exclusively from advertising. This change is a bit more difficult to explain.  What qualities do the images from advertising possess that other images do not?  As I listen to men and women discuss their collages, the answer becomes obvious.  Advertising magnifies and simplifies the representation of emotions.  Its images are designed so that consumers “get it.”  I can only conclude that generations raised on the media can understand their own emotions only through the intensifying lens of advertising.

• The collages also reveal what is a complementary trend.  The Internet, in effect, has short-circuited creative thinking.  I used to see collage creation as a serendipitous process in which my participants wandered through magazines or their homes and happened upon pictures and objects that triggered a response they might not otherwise have felt.
  
  Now, however, collages have become more logical and literal.  I see a trophy in a collage, and I ask its creator to explain it.  Oh, he says, I felt successful, so I Googled “trophy” and printed the first picture I found.  So the search for images had become less about metaphorical and and more interpretation.
  
  I can’t help but feel that this presents a problem for my collage exercise.  The Internet enables consumers to get THE answer, and that answer more often than not is verbal and logical.  The emotional dimension is lost.

What this all suggests is that in the age of the Internet, with Millennials as targets, we need new ways to tap into the emotions of consumers.  Or, we ourselves need new metaphors for that emotional response.

By Robert Relihan, Senior Vice President

Last week, I saw David Sirota discuss his book, Back to Our Future: How the 1980s Explains the World We Live in Now - Our Culture, Our Politics, Our Everything. His thesis is that, beginning in the ‘80s, American society has become increasingly focused on the individual.  He, of course, used words such as “narcissism” to describe this.  One piece of evidence he offered was that the number of people who were members of a civic organization had almost halved over a 15 year period beginning in the early ‘90s. 

Well, we have heard this all before; it’s the “Bowling Alone” argument.

But, as I listened this time, my first reaction, probably influenced by all of the online research C+R has been conducting of late, was to think, “Hey, wait a minute.  What about all of those online communities and social networks?  Aren’t communities both growing and proliferating?  Isn’t there lots of interaction among the members of these communities?  Hasn’t our ‘social capital’ merely moved online?  Isn’t the tendency of people to select their own affinity groups a replacement for traditional communities?”

The answer is yes and no.  It is important for marketers to keep in mind the differences between online communities and physical communities when they plan their strategies and conduct their research.

• Online communities are not physical; they do not have locations.  This observation may be in the category of “Duh.”  But, what does the difference mean?  We conduct surveys that use a “nationally representative sample.”  That sample reflects general population distribution.  Perhaps, it is more important to reflect the density of different self-selected communities.  A traditional qualitative project might be conducted in different markets to achieve a “national representation.”  Might it be better to be in a single market, but reflect different communities — evangelical Christians, environmentalist, and the like — in separate focus groups?  And, most obviously, if online communities are the way people are organizing themselves, shouldn’t we really be talking to consumers online?
• Online communities are more homogeneous.  Members of online communities consciously select themselves.  They seek members with whom they share beliefs and interests.  And, if that is the case, do my samples of members of these online communities need to be as large as a sample of a physical community?
• Online communities are more fragile, less stable.  Recent news suggests that Facebook traffic is declining.  There are a number of ways to interpret this data, but the trend highlights the fact that people enter and leave online communities with much greater frequency than they enter and leave physical communities.  I might have confidence in the results I obtained from a well-designed survey of my town for three years.  But, if I were to rely on a survey of an online community, I might want to revisit my results in half that time.

Online communities are the new reality.  They are indeed a rich, focused source of information.  But, a changed world requires changed methods and perspectives, and C+R is prepared to guide you through this new territory.

By Shaili Bhatt, Senior Analyst

Event coverage of the following session from the 2011 IIR Technology Driven Market Research Event (TDMR) in Chicago, May 2-3, 2011 #TDMR: “Crowdsourcing Your Research as Co-Creation” presented by Kevin Lonnie of KL Communications and Miguel Ares of Bloomberg.

As a fellow Fantasy Sports enthusiast, Kevin Lonnie and Miguel Ares’ TDMR presentation on the use of crowdsourcing a new fantasy football platform for Bloomberg Sports was like getting a backstage pass into the locker room of a big game. (Bloomberg is apparently a resource for much more than financial news!)

Crowdsourcing is an iterative ideation quali-quant method where a crowd is challenged to solve your problem, allowing market research to provide “illumination” rather than “support.” (This invalidates a quote by David Ogilvy involving drunks and lampposts).

Crowdsource the Issue, Weave the Narrative

With crowdsourcing, Kevin encourages the use of “crowdweaving”— the action of weaving the voice of the consumer into collaborative, challenge-solving initiatives.  An exemplary innovation model for crowdweaving is “Idea Sculpting,” where members are first asked to articulate their understanding of the challenge before they present their ideas for solving it.

As Kevin described it, “the race starts with everyone.”  They select the top 3-5 entries, and the challenge is to improve the concepts over a six-week process.  Good ideas transform into better ideas, until they find themselves at the threshold of a “Great Idea.”

Superior analytics have been the foundation for Bloomberg’s tools, including its Fantasy Baseball tool in 2010.  According to Miguel Ares’ from Bloomberg, however, player engagement and retention was not as high as expected.  The question was, “What is the level of satisfaction that the customer is really looking for in the world of Fantasy Sports?”

For Fantasy Football, the objective was to create a new platform to improve the user interface and experience with constant consumer feedback.  Crowdsourcing during game play was right on target!  Kevin also stressed that it is critical to involve all key stakeholders in the process so that the resulting ideas are shared and jointly owned.

Who is crowd-worthy?

This begs the question: “How do we identify who we want to speak with?” Miguel suggests not to pick professional experts, per se, but instead to pick people who think highly of the brand and really want to make a difference.  The crowd works directly with the programmers and developers—an important step for the literal metamorphosis of the ideas—in order to articulate the final product.

The researchers were able to use the crowd to tell them what they needed, and the programmers streamlined the app and the Fantasy Sports experience with this real-time feedback.  According to Kevin and Miguel, “It was a bulletin board on steroids…  In this world, the crowd rules,” and in the end, Bloomberg wins and the customers win.

By Anne Wall, Senior Vice President

The research/consumer insights industry has developed a slew of new tools that harness both technology and the ways people currently interact with one another, with products and with media.  We have been buzzing about online communities, mobile surveys, crowdsourcing, video journals, neuroscience, social media, text analytics, social gaming and more. 

These are important and necessary tools, but, they are only tools.  The community needs to continue to design research around business objectives and not around cool new tools. The recent Technology-Driven Market Research Conference was an ironic illustration of the ways in which we’re not there yet.

Conference attendees saw PowerPoint presentations that were nothing but pages of numbers and text – no audio, no video — in fact, no movement on the screen at all.  Continued failure to engage our audiences is providing our competitors from outside the research community with opportunities to move right in.

Our insights and implications need to speak to the heart of the business decisions being made.  It’s been said before, and for many years, but I’ll say it again: pages and pages of research results don’t cut it.  And, a “technology-driven” conference should have been a showcase for presentations demonstrating a visually engaging, interactive, multimedia approach. 

By Robert Relihan, Senior Vice President

If you do focus groups long enough, you will end up having to defend them.  In fact, you will end up having to defend them many times.  And, so I was sitting this week listening to someone who was tired of focus groups.  He wanted something new, different, something that put him in touch with real people.

Much of what passes for focus group criticism is simply wrong-headed; it is based on poorly conducted and poorly interpreted research.  But, it is also fair to respond to the notion that focus groups are tired and old, that fresh insight require fresh methods.

Just like a good bath, everyone needs something new once in a while, but let me point out the three things that only focus groups can do.

• Embrace the debate.  Don’t worry about the one guy who dominated the group.  The world is awash in conflicting messages.  If your idea can’t stand the assault in a group, how well will it do in the real world?  Last night I observed seven people who praised the taste and quality of one product be brought back down to earth by that one woman who said all she cared about was price. Perhaps, that’s the right proportion — one price message out weighs seven quality claims.  My client and I certainly will be thinking about it.
  
  And, remember that an effective moderator can stimulate this kind of back and forth.  No other method yields this kind of debate.

• Embrace the artificiality.  When you are in someone’s home watching them prepare dinner, only you can see what they are doing.  You are stuck with their reality.  It can be marvelously illuminating.  But, within reason you cannot swap out the entree on the fly.  You can’t see what isn’t there.  You can’t understand the meal ritual without seeing the meal ritual.
  
  But, in a focus group I can use a bit of misdirection.  I can turn what I really care about into a dependent variable.  I can present packaging variations and have consumers taste the different product (all the same, of course).  They discuss the “different” taste experiences.  Voila, they have distinguished among packaging variations without knowing that was my purpose.

• Embrace the chaos.  My last group in a series is almost never like my first.  Part of this is simple mechanics.  I learn the questions that work and the questions that fall flat.  I pick up on consumer language and integrate it into my probes.  But the real source of the change from beginning to end is that the team is constantly thinking and retooling our hypotheses and stimulus.  Concepts are revised.  New ones are created.  To be sure, this is more productive.  We are not simply collecting data, amassing observations.  We are growing and changing.

If you accept these three unique qualities of a focus group, you will be well rewarded and realize the well-conducted focus group study will always have a place in your research toolbox.

By Walt Dickie, Executive Vice President

The long strange trip from the facility to MROCs

I’ve been enjoying Robert Markley’s, Dying Planet, a terrific history of the narrative links uniting the Mars of science fiction writers and the Mars of scientists. And suddenly – as I was reading something that Michael Malin, the director of the Mars Orbital Camera (MOC) team for the Mars Global Surveyor mission, wrote about his difficulties, as a trained a geologist, in doing “geology” on Mars through the medium of images photographed from orbit – I found myself squarely back in the world of marketing research thinking about focus group moderators and clients dealing with online qualitative platforms.

Geologists think with their senses. They learn a landscape by experiencing it. Malin writes, “For field geologists, the study of an environment depends on hiking around, breaking open rocks, and seeing and touching the ground.” I know this is right; I’ve read a fair amount of writing by geologists over the years. They live to experience what Malin calls the “size, shape, texture, color, pattern, relief” of the rocks. I’ve read more than one geologist talking about licking rock dust left behind after a blow of a hammer. But the reality of what they were talking about just never really hit me before.

Thinking about Malin straining to “do geology” through the fantastic abstraction of a blurry photograph where each pixel represented 1.4 meters of what might have been real, rough rock, hefted and caressed, made me suddenly feel the plight of clients and focus group moderators making the journey from the facility to the world of online qualitative.

Moderators experience the “size, shape, texture, color, pattern, (and) relief” of their data just like geologists, and clients have always shared that experience from behind the mirror. They remember what was said – what was meant – and make connections by remembering the face and manner of the respondent who sat in the third chair to left in the facility in Pittsburgh. Without that experience, and that memory, what she said is all just dry words on a page. Or, text on a computer screen – a “response” to a “probe” in an online discussion. It has “content,” but no taste.

This seems to be one of the biggest challenges for online qualitative right now: to provide that feeling of experiencing the reality of people through the medium of text, pictures, and videos taken out of the physical, touchable context of face-to-face interaction and reduced to an abstract display of “qualitative data.”

But there’s a happy ending to the story of Malin, his MOC team, and geologists as a group. And I think there will be a similar happy ending for marketing research.

Two things have happened for the geologists. First, geology-by-remote-sensing has matured into its own specialty. As the subject matter changed from rocks to pictures of rocks, new analytic methods developed, and both newer and older generations of geologists became comfortable and skilled with them. Malin’s team proved, with their remote pictures of Martian rocks, that there had been active geological processes at work altering the Martian surface in the recent past, something that astronomers had been arguing about for a century. And planetary geology is a hot field at the moment.

The other thing that happened was that re-capturing the field geologist’s perspective on the surface of a remote planet became increasingly possible. The MOC team was working with blurry pixilated images, but they were much, much sharper than the images from earlier missions. And the newer stuff is better still. Although no human geologist has ever experienced the heft of a Martian rock, the Mars rovers, Spirit and Opportunity, gave millions a geologist’s eye view the planet’s surface. It’s telling that the NASA press release for the earlier Pathfinder mission described the Sojourner rover as a “twelve-inch tall geologist.”  

I think that similar advances will happen in marketing research. A methodology for “remote sensing” of qualitative information will mature and both clients and analysts will become increasingly familiar with it and increasingly appreciative of the vistas it opens. And we’ll also get better at capturing the feeling of immediacy in the ways we capture information online and the way we present it.

Just as Mars was simply too good for guys like Malin to pass up simply because it lacked the experiential immediacy of field geology, online qualitative is just too exciting, and the data is just too fascinating to pass up because we’re momentarily feeling the loss of the lady in the third chair to left in the facility in Pittsburgh.

By Robert Relihan, Senior Vice President

…is that they get it so wrong.  I just read another indictment of marketing research as that great stifle of creativity and innovation.  Trotted out were those ever popular examples of the focus group dominator and the poorly selected sample that both lead marketers to make bad decisions.  I have heard these stories so many times that I am convinced they are urban legends.  I wouldn’t have been surprised if a third example in the piece had been about Chicago’s most famous ghost, Resurrection Mary, directing a media plan.

Would anyone really adopt a package design because one person in one focus group really liked it?  Are business plans ever driven by a customer survey with a poorly designed sample?  Well…  But, this isn’t about research; it’s about bad research.  It isn’t about decision making; it’s about poor decision making.

Complaints about marketing research always seem to emanate from the perfect storm of poorly designed research and uninquisitive managers.  So, the next time you read someone telling you to be skeptical of research, look at the examples:

• Is there a hypothesis in the house?  Not to sound hopelessly fussy, but criticism of research with examples that never seem to have hypotheses can’t be about serious research.  Without hypotheses, any conclusion is possible, and no discipline is applied to decision making.
• Research never “says “what to do.  Criticism of marketing research always contains some phrase like “the research said.”  Research may be actionable, but it never demands action.  Research provides thoughtful managers evidence from which they can draw conclusions on which action can be based.
• Analysis speaks, not consumers.  These critiques of research are often couched in the terms of the consumer voice.  “But, in the research consumers said…”  If you wish to listen to consumers, go to a neighborhood barbeque and act upon what you hear at your peril.  Good research provides the discipline and structures to help us recognize what consumers mean beneath the chaos of what they “say.”  Don’t confuse listening to consumers with understanding them.

So, remember these guidelines the next time you hear someone criticize marketing research.  In all likelihood, he is complaining about “bad” research, or some fantasy of bad research a million miles from reality.