EmergeSmarter Blog

By Shaili Bhatt, Senior Analyst

Projective questions from in-person interviewing flow smoothly into online qualitative activities! Nancy Hardwick gave a wonderful presentation on projectives at the 2011 QRCA Conference in Las Vegas, and here we explore how these activities work well online.

Most, if not all, of my favorite projective activities in new qualitative research are derived from traditional qualitative research methods.

Online discussions or communities feature an extended interviewing phase (multiple days, weeks or months beyond traditional 2-hour focus groups), which essentially provides researchers with a welcome abundance of time to harvest and probe an always-impressive incoming flow of information. How can researchers effectively utilize this time to engage online participants and immerse them in the topic at hand?

For the time being, quick “top-of-mind” free association exercises are just as important as creative projective questions, requiring online participants to reflect for a few moments (or days!) to capture a particular feeling or experience in a thoughtful post or activity page.

Written exercises like storytelling and other creative activities like collages, when used at opportune times, can be the key ingredients to insightful and interesting new qualitative research. No stimuli are required, and natural dialogue helps to tie it all together.Nancy Hardwick of Hardwick Research presented, “Projectives in Practice,” a detailed compilation of projective techniques at the 2011 QRCA Annual Conference, which was held at the luxurious Venetian & Palazzo Resort in Las Vegas.

Hardwick encouraged the audience to interact and build upon the listed activities during her in-depth, power-packed presentation. While the focus of the presentation highlighted “what works” with in-person interviewing, the ideas and energy in the room quickly catapulted this to a presentation that refreshed my perspective and sparked the most NewQual inspiration in my notebook.

(As you may know, projective techniques are subjective questions that researchers use to elicit the underlying emotions or subconscious drivers that influence choice, as an alternative or complement to asking direct questions. Many of these techniques originally stem from projective personality tests in Psychology, which were designed for people to respond to fairly nebulous, ever-inconclusive stimuli, presumably uncovering hidden emotions and internal conflicts in the process.)

Throughout the presentation, Hardwick included a steady stream of projective techniques, resulting in a compilation of audience favorites in several important categories:

• Written Exercises
• Photo/Drawing Exercises
• Sorting Exercises

The variety of projectives serve as a reminder for how many of these time-tested exercises can be incorporated into online research.

Written exercises transition smoothly into the online world.

A few of my favorite activities are as follows (sans embellishments):

• Famous Owners—Pick a favorite/popular celebrity, and describe 1-2 thoughts about who they are. Then, ask what this celebrity’s version of (insert client’s product/service) would be like? What would it look like? How it would perform, and why? Note: This is a great technique for exploring an existing client’s products or services as well as innovation and co-creation.
• Storytelling: Describe a specific experience from the last time that you…(insert scenario). What Did You Think? Say? Feel?
• Tribute/Eulogy—Pretend that (insert product/service) no longer exists. What did it accomplish? What will you miss most about it? Describe all that you feel and want to say about the product, even if you are viewing it in a new light. Note: variations on the theme, such as a “Lifetime Achievement Award” can be more attractive for certain products/services, particularly category leaders.
• Picture Your World—Pick a picture/color that represents how you feel about (insert product/service). How does this picture/color represent how they feel, and why?

There’s no need to reinvent the wheel.

Very often, writing a discussion guide for online research is about applying techniques that are already available and crafting projective activities to capture the level of insights you can hypothesize…and then some!Hardwick advises to begin with a written exercise “early on” in the discussion, which also works well with new qualitative –even if it’s typed; this early activity serves to challenge the participants to think independently and also allows them to “own” their ideas and suggestions. A well-executed written activity that provokes thoughtful posts is also a great way to encourage insightful group discussions.Online discussion guides written like quantitative-type surveys, with numerous objective questions, can certainly be cringe-worthy. Please do not be afraid to ask creative in-depth questions: the online medium suits that well!

Leave room for the unexpected when writing questions and avoid mechanical dialogue. Don’t be overly repetitious.

When using projectives, encourage honesty and spontaneity while asking for details. I like to throw in a few natural quips at the end of questions, for example, “It could be anything!” (As in, “What’s your favorite part about ______? It could be anything!”) It is interesting to note how such a brief invitation can lead to a wider variety of posts. Treat it like a conversation, because it is!

By Robert Relihan, Senior Vice President

Those of us involved in marketing and exploring how consumers make the decisions they do should always pay attention to new approaches and constructs.  If we didn’t, we would still be wandering cities, clipboards in hand, conducting man-on-the street interviews.  So, I am extremely interested in the growth and impact of neuroscience on marketing research.

In my effort to get up to speed in this area I have found Roger Dooley’s website and blog particularly useful.  He covers the breadth of the topic although there are fairly frequent references to those who question the approach, labeled “alarmists.”  And, it is true that there is a good deal of overheated reactions to the idea of marketers probing our brains for ways to make us buy products without realizing we want them.

But, there is reasonable questioning of neuromarketing, and in a recent article in the Wall Street Journal, “Brain Scan Overload” by Jonah Lehrer is an excellent example.  He focuses on the use of functional magnetic resonance imaging, or fMRI, a device used to provide a picture of the brain as it reacts to different stimuli.  He points out several ways in which the use of the device in such research might be questionable:

• It uses blood flow as a surrogate for the activity of neurons in the brain.
• There are complex algorithms that separate the noise from the signal.  The result can be a simple picture of what might be a very dense psychological state, such as happiness.
• Various areas of the brain play a role in multiple emotions.  So, the insula plays a role in love, disgust, and bodily pain.  Lehrer points to research that associated a spike in activity in that region of the brain with love for the iPhone.  Why not disgust or pain?

But at the very end of his article, Lehrer makes a point about neuroscience that gives me pause.

“What's worse, the very fact that we're looking at a brain scan seems to inhibit our critical thinking. Deena Skolnick Weisberg, a psychologist at Temple University, has demonstrated that merely referencing fMRI research can bias the evaluation of scientific papers.

“When she gave neuroscience students and ordinary adults a few examples of obviously flawed scientific explanations, people were consistently able to find the flaws. However, when these same explanations were prefaced with the phrase ‘Brain scans indicate,’ both the students and adults became much less critical.”

By Robert Relihan, Senior Vice President

The pace at which smartphones and other mobile devices are coming to define how we interact and communicate is amazing. We have noted it before and we see the impact intensifying where ever we look.

• The US Army is experimenting with smartphones in combat.  In the past, the military has spurred the development of communications devices, hardened for battle.  But, the economy and ubiquity of smartphones are just too enticing.  What is more, the familiarity of virtually every young soldier with these devices makes training and integration seamless.

But, using smartphones has the potential for fundamentally changing military culture.   Individual soldiers on the battlefield can receive real-time intelligence unmediated by the traditional chain of command.  Each soldier can know immediately the location of friendly forces as well as the enemy.  Information can be distributed instantly among all the soldiers in an area.  Hierarchies are flattened and communication is effectively two-way.

• On a lighter note, Rebecca Greenfield notes how the smartphone has eliminated the need for all sorts of devices we used to regard as essential — cameras, watches, iPods, wallets.  She calls this the “Smartphonication of Stuff.”  And, while we have had cameras and watches for some time, the iPod (and even its ancestor the Walkman) is comparatively recent.  The smartphone has rendered it obsolete in the space of a mere generation.
• Finally, as an example of how culture is bending to the ethos of smartphones, a theater in Seattle is planning to allow patrons use their phones during performances.  No more requests that all phones and recording devices be turned off.  As the theater’s executive director said, “Simply forbidding it and embarrassing people is not the way to go.

So we are wiring the building in anticipation of finding ways to make it work over time."

We all know that the march of mobile communications will change fundamentally how we conduct research, but these stories suggest how significant that change can be.

• Many of us can be quite enthusiastic about the ability of consumers to send their “answers” to us wherever they are.  We can eavesdrop on their reactions in the moment.  But, just like the soldiers in the army’s experiment, they will want to see the hierarchy flatten; they will expect more control.  Consumers will demand to frame the questions as well as provide the responses.
• The smartphone may have rendered the camera and the watch obsolete.  For many people, it will render the computer obsolete.  Reflect on how quickly netbooks faded from the scene.  We already had a much better “small computer,” our phones.  There was a time when any project began with the question, “Should we do it on-line or over the phone?”  Now we are asking, “Should we have a mobile component in the project?”  In a few years, we will simply take for granted that our consumers are answering on their mobile devices.
• We will need to engage consumers in a way that makes sense in the culture of the smartphone and the social network.  No longer can we expect to bend them to our style and approach.  Consumers accepted the “social science survey” and the “focus group” when the one-way interview defined the environment.  But, when environment is structured by the shout-out — “Who knows a great place for dinner and meet me there?” — a semantic differential just won’t cut it.

To be successful in the future, to be able to hear the voices of consumers in all their tones and timbres, we will need to adjust our ears to the new reality.  In fact, we are likely to become more listeners and collaborators and less askers and analyzers.  As the Seattle theater director said, we have to find “ways to make it work.”

By Walt Dickie, Executive Vice President

When my two college-age sons were younger they each went through a period of fascination with what I think of as “alternate reality” or “science fiction presented as fact” cable TV programs. Our house was awash for years in aliens hidden in secret government facilities, Sasquatch sightings, and paranormal activities of all kinds.

None of this bothered me too much. Although the sci-fi-as-sci-fact genre hadn’t been so popular when I was a kid, I had loved Erich von Dänigan, and devoured everything I could find about Bigfoot – once even giving serious thought (for a week or two) to writing my Ph.D. dissertation about Bigfoot lore before coming to earth on the realization that my paltry linguistic skills weren’t up to the challenge of the Northwest Coast languages.

But my kids were getting all this via cable at a much younger age than I was, and they didn’t have the grasp on the “science” part yet. I had to have some way of talking to them about what they were seeing that admitted its attraction while warning them about taking it all too literally.

Talking to my older son one day about extraterrestrial visitors, and struggling with the knowledge that the stories of little green men that he found so fascinating certainly had some scientific plausibility, I told him that I thought it would be so cool if it was true. And I saw that what I was trying to say had finally clicked for him. Of course he was fascinated, of course people wrote about this, went to conventions, and searched the skies (and their backyards) for aliens because it would be so cool if it was true. Once the aliens, Yetis, time travel, ghost sightings, and all the rest were seen as powerfully compelling stories, he got the point I had been trying to make about his fascination with them and my parental concern.

That experience had a real effect on me; it was the first time that I fully realized how our entertainment-driven culture has become fascinated by things that are cool to think about, even when we know at some level that they aren’t true. It’s not that we value illusions of truth, it’s that we value cool so highly. This is where internet memes are born – internet flash mobs playing all the variations on a theme that is experiencing its moment of coolness.

I think about that experience a lot these days; only I’m not thinking about possible remnant populations of otherwise extinct giant apes wandering around barefoot in the Himalayas. I’m thinking about the unceasing drumbeat of blog posts and articles hawking the embrace of all-out innovation as the only means of escape from certain commercial death.

“Innovate or die” has been with us since the tech boom of the 90s, and it has evolved from a taunt thrown out by a few hard-charging internet entrepreneurs to well-nigh the accepted gospel of modern business. It seems to be especially popular among bloggers and other bystanders; a hair trigger response to any commercial stumble, good for a quick post requiring little thought. Lately it’s become the war cry of the marketing research commentariat, who would have us toss off the shackles of hide bound, obsolete, research techniques – apparently abandoning clients and revenue streams in our haste to get out of long-standing but soon extinct business lines.

C+R and I both survived the 90s – by innovating – and I’ve had a lot of time to think about the events of those years and what they mean to our future. Our experience put me firmly in the pro-innovation camp, and I thought I was comfortable there. Then, a few years ago, my college alumni magazine, Technology Review, which has become a fairly successful commercial enterprise itself, announced that “innovation” would henceforth its theme, its be-all and end-all.

I found that strangely troubling. I’m an MIT grad, but I am not now nor have I ever been an engineer, scientist, or even a technologist in any very serious sense. I’m an anthropologist by professional training, and what technical expertise I have has always been harnessed to business issues.

But when Technology Review started to really harp on “innovation,” I found myself wondering about all of the engineers who had spent their careers improving processes, increasing the accuracy and speed of measurements, making things more efficient, safer, or economical. Certainly those achievements required something reasonably called “innovation,” but that kind of innovation wasn’t what the “innovate or die” crowd were talking about.

You don’t “innovate or die” just by improving something. You invent something “revolutionary.” You “change the world.” You overthrow the old and embrace the new. You cause a “paradigm shift.”

I have a visceral understanding of the allure of paradigm shifts. The voice of Thomas Kuhn, extolling the concept of “paradigm shifts” in the history of science during the 60s, sang as sweet a song in the corridors of academe as The Beatles or The Dead. We wanted to get high, get laid, and cause a paradigm shift that would overthrow the stodgy dogma of whatever we were majoring in – not necessarily in that order. I remember it well. And, like most young academics, I loved the smell of it in the morning.

So why, I found myself asking, does it bother me, 40 or 50 years later, that my alumni magazine has gone all innovation on me? Was I now backing the fuddy duddies? This question really bothered me, raising the possibility, as it did, that I was not only backing them but also joining them.

In the past few years I think I’ve finally figured it out. Kuhn was writing about big-S Science, and the Tech Review has chosen to concentrate more on Big-T Technology rather than the small-t technologists it used to write about. Paradigms are the right thing to think about when you’re talking about big-S, big-T stuff. I thought so in the 70s, and I think so now.

But if you’re a small-letter scientist, technologist, or marketing researcher, then the issue isn’t as clear. Should you, in fact, stop wasting your time on something that will be displaced by a paradigm shift?

Let’s imagine you live near Lake Michigan, as I do. It’s summer, it’s warm, and the sun is shining. The gorgeous blue waters and fresh breezes beckon. You know, of course, that the paradigm of gorgeous weather will shift, possibly very soon. And, this summer at least, you have every reason to believe that 60 mph winds and 20’ waves may make that shift very unpleasant indeed. For purposes of this little example, you have no access to any weather forecast of any kind. What do you do? Stay on shore or get in a boat?

The fact is that, until the advent of modern computerized weather models, no forecasting tools could consistently beat the prediction that tomorrow’s weather will be very much like today’s. Your best bet, then, in the absence of better information like an online satellite view, is to predict a continuation of more of the same. Shove off, hoist the sails, and enjoy the continuing lovely weather. At least in the short run.

Let’s try another analogy. You’re a marketing research company that has a good-sized survey research business. You’ve been doing well on survey research for many years, you negotiated the transition from phone and mall to the internet, and you’re doing well getting your head and hands around the mobile revolution.

But you see a paradigm shift coming. Big data could shift the paradigm and undermine the market for survey research because, given enough data, what people actually do as reflected in their searching, shopping, purchasing, friending, tweeting, and other choices on millions of web sites is a much more reliable indicator of their future behavior and their opinions. And a paradigm shift in cognitive science says that people don’t even know what they’re thinking when they’re thinking it; that what they think they think and whatever “thought” it is that drives action in their world are two totally different things, often in conflict with one another. Who will care about what people imagine they think in the future? Your survey research could be shifted right into the trash.

Or will your recent good results from survey research continue, like the weather, at least in the short run? For that matter, even if your business tanks in the long run, what exactly does “long” mean? Do you get out of the business today? Next year? Suddenly? Gradually? And what does “out” mean? Quit? Put less emphasis on selling and marketing? Stop developing completely and freeze everything just as it is today?

Since we’re talking here about you, a small-m marketing researcher, not the big-M marketing research industry as a whole, it’s pretty hard to know the right answer. Certainly these, and other equally large, evident paradigm shifts, are going to impact Marketing Research – or Consumer Insights, Decision Guidance, or whatever we wind up calling what we do. But what happens at big-M scale may not be what happens in your immediate neighborhood.

Famously, buggy whips went out a century or more ago, killed by the big paradigm shift to internal combustion engines, automobiles, and global warming. But there are still buggy whips being made and sold, and someone is probably making a nice living off them. Along about the same time that the horse-and-buggy went out, a huge industry based on fancy feathers for ladies’ hats, that had shipped tons upon tons of feathers annually, died out, swamped by a sea change in fashion. But there are still feather merchants to be found, and their industry, once dominated by sales to fly fishermen and Vegas-style costumers, is booming thanks to a fad for hairdos incorporating feathers. I remember reading a story a few years ago about a local firm’s turnaround – they had gotten out of their struggling “modern” electronics business and established themselves as the dominant player in the global market for replacement vacuum tubes for legacy equipment and were making record profits! Vacuum tubes! Who knew? And closer to home, there are still many phone rooms plying their trade for marketing research long after the online paradigm shift moved the industry in a new direction.

Let me be clear: I’m not arguing that it’s better, wiser, or more lucrative for your career or business to live on the backside of a paradigm shift, only that there are a lot more choices and possibilities than just “get with the new paradigm” or “drop dead.” Anyone who tells you otherwise has either never managed a real business or going for sensationalism over sense. Hooey!

It’s pretty clear why we’re so enamored of “innovate or die”– you don’t have to have paid much attention in any of your lib arts courses to recognize and understand the appeal of a story arc. We humans all crave stories and are fascinated by them, so condensing the complexities of actual life into the simple narrative of “innovate or die” is perfectly understandable, even if pretty much useless as advice. Innovators on one side – successful, profitable, with it, and future-oriented – and dead companies – foggies, fools, and bankrupt stick-in-the muds – on the other!

It is, I think, a winner take all fable for a winner take all age.  It would just be so cool if it was true!

By Robert Relihan, Senior Vice President

Those of us who face the daunting task of sifting through mounds of verbal data — forty individual interviews or a three week on-line community — synthesizing all of those words, and presenting our analysis clearly, succinctly, AND with impact have been intrigued by word clouds.  They seem simple and elegant; they reduce all of those words to a picture of the important themes.  Now, there are numerous web sites to help us create word clouds, Wordle being only the best known.

So, why do I feel vaguely dissatisfied when I look at word clouds, even the ones I have created?  They often seem to miss the point or be overly simplistic.  Creating a word cloud sometimes feels like I have cheated myself and my audience.

Jacob Harris of The New York Times makes the case against word clouds and, in the process, gives a brief primer on how to report its data.  It is an incredibly useful article.

He begins with a concept that should become a mantra to market research professionals.  The critique of data clouds is based on the principles of data journalism.  When many of us began our careers, the model for reporting was the academic paper.  I wrote reports a long time ago with footnotes.  Now we strive for clarity and simplicity.  The magazine or newspaper (on-line versions, of course) is our guide.  Data journalism should be our art.

“Visualization is reporting, with many of the same elements that would make a traditional story effective: a narrative that pares away extraneous information to find a story in the data; context to help the reader understand the basics of the subject; interviewing the data to find its flaws and be sure of our conclusions.  Prettiness is a bonus; if it obliterates the ability to read the story of the visualization, it’s not worth adding some wild new style or strange interface.”

The ways Harris point out how word clouds go wrong provide us with a road map for good reporting or, rather, good data journalism.

• Word clouds are based on a very rudimentary textual analysis.  In most cases, a phrase-level or a thematic analysis would provide richer and more penetrating analysis.  The general lesson of this observation is that we need to focus on the concepts that knit the words together consumers use and not on the words.
• Word clouds are often used when textual analysis is not the appropriate tool.  As Harris says, in our analysis we should not confuse “signifiers with what they signify.”  We need to use the appropriate methods for getting below the surface of consumers’ comment.  Simply digesting their words will not do that.
• Word clouds have a dirty secret.  They really aren’t analysis.  They leave readers with the task of peering at the image and discerning the meaning themselves.  Word clouds make the assumption that the meaning is obvious.  But, any analysis worth its salt requires some explanation; it requires framing and focus.
• Finally word clouds miss the narrative.  I am not saying what we write or present should be long, dreary marches through the data.  Hardly.  We need to find the thread or threads that bring fresh insight to a particular area of consumer behavior.  There are often several reasons why two words might dominate a word cloud.  We need to create the story for the reader or listener that that makes just one of these reasons the most compelling and the most relevant.  And, incidentally, that narrative still can be visual.

A word cloud tries to make us believe its immediacy produces insight when, in fact, it may mask the narrative we “data journalists” should be creating.

By Walt Dickie, Executive Vice President

Mobile. The next big thing, right? Well, maybe or maybe not. I’ve just been Googling mobile marketing research topics, downing a few white papers, and reading some conference presentations, and it strikes me that an awful lot of marketing researchers simply don’t understand what mobile means for the research business or the impact it’s going to have.

Here’s the problem: Imagine you’re a buyer of marketing research. Today. Now. You’ve got more questions and issues than you can easily cram into your limited budget and timeline. You’ve worked up a survey research design, and you’re balancing sample costs against the sample sizes you need for the analysis.

Then someone points out that some of your key segments are going to be pretty difficult to get. People who are very mobile. People who don’t have home computers and internet connections. “But they have cell phones,” says someone else. “We could maybe interview them on their phones!” Briefly, your spirits soar – you’ll add an online sub-sample!

Then the realities hit: short interviews, something called a “mobile template,” increased sample costs, a whole new technology to tackle, and a methodological nightmare to merge the mobile data and findings with the rest of the project. The benefits aren’t enough to justify the approach. “Great idea, though. We’ll have to keep that in mind.”

This is totally wrong. It’s not just wrong today, for today’s project – you almost certainly made the right choice for your already over-burdened, under-funded current project. But it’s wrong for tomorrow, and wrong for the industry.

Because mobile isn’t simply a matter of sample. Mobile is about culture. More specifically, it’s about culture change. And you’re very likely to be part of that change already.

Don’t believe me? Do you have a smartphone? Have you had it for at least six months? OK – answer this question: Out of the last ten times you checked your email, how many times did you use your iPhone, Blackberry, or Android phone? Office computer? Home computer? Tablet?

I’ve been paying attention to my own behavior for the last week or so, and I’d guess that out of ten tries, I check email 4 times on my phone, 5 times on my office computer, once on my tablet, and I probably didn’t use my home computer at all.

Now think about those times when you used your phone: Where were you? For me, I check my email using my phone from lots of places: sitting at home with the phone at my side, driving to or from work, running errands on weekends, in the office when I’m away from my desk. And that’s where the culture shift lies.

I have my phone on me most of the time – it’s sitting on my desk as I write this on my office computer. And I use it almost everywhere, including a few times when I’ve already got Outlook running on a computer right in front of me. I’ll bet you do the same.

My phone is with me most of the time. It’s my personal device, the one I have close to me no matter where I am or what I’m doing.

The smartphone is the logical culmination of what people talked about in the 80s and 90s when they got enthused about how personal the desktop computer was. Here, for the first time, were computers designed to be used by a single person, with all kinds of ways that they could be customized to that person’s tastes. Screensavers, wallpaper, custom menus and button bars, programs you liked and used – all eons away from the big iron where you might have had a timesharing account.

The smartphone is the most personal PC yet. The longer you have one, the more you find yourself using it, often for things you thought you’d never do on such a small device, in places where you never used your previous “not”-smart phone. Over time, you find yourself expecting to use it for just about anything that you can use it for. And you’re increasingly annoyed when you can’t – when a web site doesn’t render using a mobile template and forces you to endure a nightmare of zooming and hitting links inadvertently because it’s cramped, crowded layout is hostile to your little screen and big fingers. Or when a survey invitation dumps you into a questionnaire clearly designed with the expectation that you’re sitting at a keyboard, looking at a big screen, and navigating with a mouse. When you’re not in the “mobile sub-sample” because the marketing research industry didn’t get how the culture changed when smartphones appeared in force and everyone used them for everything. Because MR got mobile wrong.

By Robert Relihan, Senior Vice President

If you have conducted focus groups, or even observed them, you have probably noticed that the energy level can vary over the course of a day.  I have always taken this as the normal course of events. But, it turns out there is a rather interesting explanation for this ebb and flow — decision fatigue.  A recent article by John Tierney in the New York Times, “Do You Suffer From Decision Fatigue?,” describes the research surrounding this phenomenon in great detail.

I had always thought people simply get tired during the course of the day, but Tierney demonstrates that the mere act of making decisions wears us down, making us less able make additional choices.  He points to a study in which individuals who simply review material were better able to make choices about it at the end than those who were forced to repeat incremental choices. 

He also notes that those who receive food make better decisions.  And, it is not the psychological reward of the food that works.  Tasteless, but sugared grub will have more effect than a sugar-free sundae.

What is the impact of decision fatigued?  One makes poor decisions, of course.  But, what is a poor decision?  The fatigued person defaults to the familiar, the usual, the expected.  In other words, he or she avoids making the hard choices — just those choices we would like research participants to make.

So, what is the lesson here for researchers?  Well, I am going to take the cans of Pepsi and cookies on the table much more seriously than I have in the past.  I might even ban water and diet beverages.  On a more fundamental level, it has caused me to re-think the number and nature of exercises I ask people to perform in a focus group.  Let them create a collage at home rather than force them to select pictures for a collage during the interview.  Might there be implications for questionnaire design?

Concerns about decision fatigue also make on-line qualitative and MROCs look all the more promising.  Participants can respond to our tasks at the times they are freshest and best able to evaluate whatever it is we wish them to judge.

There is another troubling issue Tierney raises.  “Decision fatigue is a major — and hitherto ignored — factor in trapping people in poverty. Because their financial situation forces them to make so many trade-offs, they have less willpower to devote to school, work and other activities that might get them into the middle class….  A trip to the supermarket induces more decision fatigue in the poor than in the rich — because each purchase requires more mental trade-offs.” At first this seems like an important, but a political concern.  Yet, it has implications for those who wish to understand the behavior of consumers.

Dan Peck’ Atlantic article, “Can the Middle Class be Saved?” described what he calls the “hollowing out” of the American middle class.  His economic arguments are familiar. Over the past several decades, a larger proportion of the nation’s wealth has accrued to a smaller proportion of individuals at the top.  But, in the process, the middle class has fractured.  Now, the non-professional middle class are more likely to resemble in behavior and attitudes the “high school drop-out poor” than the “college educated members” of the middle class.

The sub-text — many more of those whom we survey or wish to have in our MROCs and focus groups are likely to be in a semi-permanent state of decision fatigue.  Our methods and approaches will need to be much more sensitive to this fact.

By Juan Ruiz, Senior Research Analyst

Caution! What you said, what you meant to say, and what the other understood may not always be the same thing…

At C+R Research, we have a diverse group. Last week, it was the birthday of a Polish colleague. A few emails started going back and forth to see where we would take him for lunch. Then, one of our Hispanic colleagues sent an email that read “Szczęśliwy dzień urodzenia.” Clever as she is, she had used an online site to translate “Happy Birthday” to Polish. That email triggered a chain of emails – all in Polish, although no one other than the birthday boy spoke Polish. Later he mentioned that many messages sounded funny because computer translations can be excessively literal.

This made me think about the challenges we face every day translating questionnaires. Our Hispanic division, LatinoEyes® specializes in the U.S. Hispanic market where most of our studies require conducting fieldwork in the respondent’s language of choice (English or Spanish). Paying special attention to our translations is crucial. If respondents answering questions in English understand the questions differently than the respondents answering in Spanish, our data and our findings will suffer.

Now, making sure that nothing is lost in translation is not an easy task.  There are words that have different meanings in different Spanish-speaking countries, and even some expressions can vary significantly by length in different dialects.  All of this adds complexity to the translation process.

How do we do it at LatinoEyes®? Through lots of communication! It is key that the person responsible for the translation is on the same page with the person who wrote the English version. The translator needs to be a speaker of the target dialect. We debate; we come to consensus.  We have Spanish speakers from different countries of origin, and they have different levels of acculturation. They all share their point of view when needed.

How are you doing it?

ByWalt Dickie, Executive Vice President

Many of you may have probably seen the recent announcement of a new report from the Pew Internet and American Life Project on cell phone usage.  Reporting of the findings has been a bit hyperventilated, focusing on odd behavior, such as turning phones off to get a break from using it (between 26% and 32% for all age groups under 65) and pretending to talk on the phone to avoid talking directly to someone (30% of 18-29 year-olds in past month).  More about this last one later.

The report, however, highlights some genuinely interesting trends.

• The increasing dominance of cell phone use generally, but especially for text messaging and photography.
• The fact that African Americans, Latinos, and to a lesser degree those with college educations, urban- and suburbanites, and parents over-index on cell usage.
• The enormous difference between “Smartphone” users and other “Cell/Feature Phone” users.
• The diminishing differences between the ways different age groups under 65 use smart phones.

With 83% of adults owning cell phones, and ¾ of them using their phones for text messaging and photography, it is clear that research techniques based on text messaging and/or picture-taking should be considered as fitting quite nicely into “normal life.”  While text messaging can be seen as roughly analogous to writing or typing answers to traditional survey instruments, “picture taking” now has a place primarily in specialized methods such as ethnographies and MROCs.  But, as research becomes increasing mobile, might not the normal way to answer many typical questions be with a picture?  What groceries did you buy this week?  What do you use to clean the floor?  What snacks do you eat regularly?  There will be no forgetting key parts of the answer when it is a picture.

It seems equally clear that Smartphone Owners (35%) are a distinct segment of the population whose use of phones for messaging, internet access, games and entertainment, and retrieval of information needed “right away” index over Cell/Feature Phone Owners by anywhere from two to five times or more.  It seems likely that we will need a new survey grammar when we create mobile instruments directed at this more impatient target who craves immediacy.  It may not be the place to ask questions about the restaurants visited six months in the past.

Smartphone owners have a distinctive demographic profile, but age is clearly becoming a salient variable – although the youngest age segments over-index on pretty much everything, all Smartphone Owners under about 50 clearly share a core set of behaviors that is distinct from the behavioral set of Cell/Feature Phone Users.

By the way, if Smartphone Users are a distinct population, how will we talk to people about cell phones vs. smartphones?  Pew doesn’t seem to have a term for devices that aren’t smartphones. When they screen, they use awkward questions like, “As I read the following list of items, please tell me if you happen to have each one, or not. Do you have...a cell phone or a Blackberry or iPhone or other device that is also a cell phone?” The industry sometimes uses “feature phone,” but I’ve never heard a normal person use that term. “Dumb phone?” My son had a pretty nice phone with a touchscreen that wasn’t an iPhone, Android, or any other obvious “Smartphone” type.  He called it “a wannabe phone.”

Back to people who pretend to make cell phone calls to avoid contact with others.  Instead of wringing our hands about the end of civilization, we need to recognize what it truly indicates.  It is interesting that this is behavior of young adults, those most at home with all things mobile.  But, also no surprise.  Cell phones truly have become rooted among young adults.  The gestures of their usage have generally accepted meanings within the “culture.”  And, nothing demonstrates this more powerfully than the fact that faking a call works.

By Lynne Bartos, Vice President

There is nothing more embarrassing for a researcher than to hear a client say “…this doesn’t really address the business questions that we set out to answer.” This is more common an occurrence in research reporting than most of us would care to admit. But unfortunately, much report writing these days falls short of expectations for those on the client side.  This is likely due to more emphasis on methodology or analytic technique at the expense of clear graphics, creative story-telling and actionable direction.

What often happens during the report-writing process is that market researchers have their direct research client in mind.  They neglect the fact that their direct contact must present these findings to the ultimate stakeholder in the process — someone in senior management or the head of marketing who does not function in the research realm. 

We need to take conscious steps to break out of our little bubble to avoid some of the lingo that is prevalent in research circles. You know what I mean if you’ve ever found yourself presenting your findings to marketing folks.  While peppering them with terms such as “mean,” “monadic,” “DK/NS,” “latent class,” and the like, you suddenly notice the deer-in-the-headlights reaction.  Worse yet, your audience’s eyes glaze over completely.  These terms are foreign to many marketers and, frankly, most of them couldn’t care less about such things.  They simply want a viable solution to the particular business need they set out to address.          

So, when writing a research report for my clients it helps me to keep a few things in mind….

Speak to Marketers in Their Language

Focus on what marketers care most about — getting customers, keeping customers, and increasing their share of the customer’s wallet.  So tell them what is meaningful to them….

• How to position their brand
• How best to price it 
• Who their best prospects are and how to reach them
• What message should they be communicating 
• Who are their most loyal and valuable customers
• How do they keep them loyal to their service or brand 

Net, net — put some Marketing-Speak into your report, and leave out the Research-speak.

Tell a good story

A good report tells a good story.  So, how do you tell a compelling story?  Start by getting organized!

• Develop an analytic plan that focuses on business issues and objectives — the questions that need to be answered. 
• Outline how the questions will be. 
• Once the data is in, all team members should know how the data relates to those question, and they can craft the best story together.

Remember, every page in the report should contribute to the story!  If something doesn’t contour well with your story, stick it in the Appendix.  How many hundred-page reports have you been subjected to where the charts are all in the same order as the questionnaire?  Where is the story?  

It’s also important to stick closely to your analytic plan when crafting your story. The analytic plan is what helps to keep everyone focused on why the research was conducted in the first place.

Insightful Headlines and Bullets

What I also find helpful in getting my arms around the story is to write effective bullets and headlines for the data presented.  Too many people think an insight is reiterating the numbers that are in the charts.  Remember, anyone can read the numbers on a chart – our job, as researchers, is to pull the deeper insights from seemingly obvious data.

Be Creative and Have a Llittle Fun

Creativity also comes into play!  Package the story in a creative way. No one wants to see rows and rows of data. Make the report visually appealing so you don’t intimidate those who are going to be using the findings to help drive strategy.  Avoid too much text and too many numbers.

And, don’t be afraid to insert some humor here and there. It reminds your clients that you are human and helps to lighten the tone and keep things relaxed.   

Get to the Heart of It

And finally, probably the hardest part of the report process for any researcher is to get straight to the heart of it… what is the story – conclusions, implications, and recommendations.  Go to the next step to tell them what the data MEANS, and what they might consider doing to maximize their investment.

And there is nothing sweeter to a market researcher’s ears than to hear a client voice saying, “Thanks, this really addresses the business questions that we set out to answer!”