EmergeSmarter Blog

Online Projectives: It Could Be Anything!

Posted on Tue, Dec 20, 2011

By Shaili Bhatt, Senior Analyst

Projective questions from in-person interviewing flow smoothly into online qualitative activities! Nancy Hardwick gave a wonderful presentation on projectives at the 2011 QRCA Conference in Las Vegas, and here we explore how these activities work well online.

Most, if not all, of my favorite projective activities in new qualitative research are derived from traditional qualitative research methods.

Online discussions or communities feature an extended interviewing phase (multiple days, weeks or months beyond traditional 2-hour focus groups), which essentially provides researchers with a welcome abundance of time to harvest and probe an always-impressive incoming flow of information. How can researchers effectively utilize this time to engage online participants and immerse them in the topic at hand?


For the time being, quick “top-of-mind” free association exercises are just as important as creative projective questions, requiring online participants to reflect for a few moments (or days!) to capture a particular feeling or experience in a thoughtful post or activity page.

Written exercises like storytelling and other creative activities like collages, when used at opportune times, can be the key ingredients to insightful and interesting new qualitative research. No stimuli are required, and natural dialogue helps to tie it all together.Nancy Hardwick of Hardwick Research presented, “Projectives in Practice,” a detailed compilation of projective techniques at the 2011 QRCA Annual Conference, which was held at the luxurious Venetian & Palazzo Resort in Las Vegas.

2011 QRCA Conference, Projectives in PracticeHardwick encouraged the audience to interact and build upon the listed activities during her in-depth, power-packed presentation. While the focus of the presentation highlighted “what works” with in-person interviewing, the ideas and energy in the room quickly catapulted this to a presentation that refreshed my perspective and sparked the most NewQual inspiration in my notebook.

(As you may know, projective techniques are subjective questions that researchers use to elicit the underlying emotions or subconscious drivers that influence choice, as an alternative or complement to asking direct questions. Many of these techniques originally stem from projective personality tests in Psychology, which were designed for people to respond to fairly nebulous, ever-inconclusive stimuli, presumably uncovering hidden emotions and internal conflicts in the process.)

Throughout the presentation, Hardwick included a steady stream of projective techniques, resulting in a compilation of audience favorites in several important categories:

  • Written Exercises
  • Photo/Drawing Exercises
  • Sorting Exercises

The variety of projectives serve as a reminder for how many of these time-tested exercises can be incorporated into online research.

Written exercises transition smoothly into the online world.

A few of my favorite activities are as follows (sans embellishments):

  • Famous Owners—Pick a favorite/popular celebrity, and describe 1-2 thoughts about who they are. Then, ask what this celebrity’s version of (insert client’s product/service) would be like? What would it look like? How it would perform, and why? Note: This is a great technique for exploring an existing client’s products or services as well as innovation and co-creation.
  • Storytelling: Describe a specific experience from the last time that you…(insert scenario). What Did You Think? Say? Feel?Storytelling Say Think Feel 540x145
  • Tribute/Eulogy—Pretend that (insert product/service) no longer exists. What did it accomplish? What will you miss most about it? Describe all that you feel and want to say about the product, even if you are viewing it in a new light. Note: variations on the theme, such as a “Lifetime Achievement Award” can be more attractive for certain products/services, particularly category leaders.
  • Picture Your World—Pick a picture/color that represents how you feel about (insert product/service). How does this picture/color represent how they feel, and why?

There’s no need to reinvent the wheel.

Very often, writing a discussion guide for online research is about applying techniques that are already available and crafting projective activities to capture the level of insights you can hypothesize…and then some!Hardwick advises to begin with a written exercise “early on” in the discussion, which also works well with new qualitative –even if it’s typed; this early activity serves to challenge the participants to think independently and also allows them to “own” their ideas and suggestions. A well-executed written activity that provokes thoughtful posts is also a great way to encourage insightful group discussions.Online discussion guides written like quantitative-type surveys, with numerous objective questions, can certainly be cringe-worthy. Please do not be afraid to ask creative in-depth questions: the online medium suits that well!

Leave room for the unexpected when writing questions and avoid mechanical dialogue. Don’t be overly repetitious.

When using projectives, encourage honesty and spontaneity while asking for details. I like to throw in a few natural quips at the end of questions, for example, “It could be anything!” (As in, “What’s your favorite part about ______? It could be anything!”) It is interesting to note how such a brief invitation can lead to a wider variety of posts. Treat it like a conversation, because it is!

Tags: market research tools, Market Research Conference, qualitative research, Online qualitative research

The best way to find out what consumers feel is just to ask them.

Posted on Mon, Dec 12, 2011

By Robert Relihan, Senior Vice President

Those of us involved in marketing and exploring how consumers make the decisions they do should always pay attention to new approaches and constructs.  If we didn’t, we would still be wandering cities, clipboards in hand, conducting man-on-the street interviews.  So, I am extremely interested in the growth and impact of neuroscience on marketing research.

In my effort to get up to speed in this area I have found Roger Dooley’s website and blog particularly useful.  He covers the breadth of the topic although there are fairly frequent references to those who question the approach, labeled “alarmists.”  And, it is true that there is a good deal of overheated reactions to the idea of marketers probing our brains for ways to make us buy products without realizing we want them.

But, there is reasonable questioning of neuromarketing, and in a recent article in the Wall Street Journal, Brain Scan Overload by Jonah Lehrer is an excellent example.  He focuses on the use of functional magnetic resonance imaging, or fMRI, a device used to provide a picture of the brain as it reacts to different stimuli.  He points out several ways in which the use of the device in such research might be questionable:neuroscience  market research

  • It uses blood flow as a surrogate for the activity of neurons in the brain.
  • There are complex algorithms that separate the noise from the signal.  The result can be a simple picture of what might be a very dense psychological state, such as happiness.
  • Various areas of the brain play a role in multiple emotions.  So, the insula plays a role in love, disgust, and bodily pain.  Lehrer points to research that associated a spike in activity in that region of the brain with love for the iPhone.  Why not disgust or pain?

But at the very end of his article, Lehrer makes a point about neuroscience that gives me pause.

“What's worse, the very fact that we're looking at a brain scan seems to inhibit our critical thinking. Deena Skolnick Weisberg, a psychologist at Temple University, has demonstrated that merely referencing fMRI research can bias the evaluation of scientific papers.

“When she gave neuroscience students and ordinary adults a few examples of obviously flawed scientific explanations, people were consistently able to find the flaws. However, when these same explanations were prefaced with the phrase ‘Brain scans indicate,’ both the students and adults became much less critical.”

Neuromarketing research seems to be another effort to find research methods that give “the answer,” unchallenged or mediated by thought and analysis.  Marketers make decisions influenced by research.  Research helps provide understanding of consumer behavior.  But, when the method is the answer, subtlety and flexibility are lost.  Ultimately, a sensitive marketer can learn a good deal and make successful decisions by listening directly to consumers.

Tags: Market Research Reporting, neuromarketing, Market Research, market research tools

Embracing the New World of Mobile Research

Posted on Fri, Dec 2, 2011

By Robert Relihan, Senior Vice President

The pace at which smartphones and other mobile devices are coming to define how we interact and communicate is amazing. We have noted it before and we see the impact intensifying where ever we look.

  • The US Army is experimenting with smartphones in combat.  In the past, the military has spurred the development of communications devices, hardened for battle.  But, the economy and ubiquity of smartphones are just too enticing.  What is more, the familiarity of virtually every young soldier with these devices makes training and integration seamless.

But, using smartphones has the potential for fundamentally changing military culture.   Individual soldiers on the battlefield can receive real-time intelligence unmediated by the traditional chain of command.  Each soldier can know immediately the location of friendly forces as well as the enemy.  Information can be distributed instantly among all the soldiers in an area.  Hierarchies are flattened and communication is effectively two-way.Mobile Research Survey

  • On a lighter note, Rebecca Greenfield notes how the smartphone has eliminated the need for all sorts of devices we used to regard as essential — cameras, watches, iPods, wallets.  She calls this the “Smartphonication of Stuff.”  And, while we have had cameras and watches for some time, the iPod (and even its ancestor the Walkman) is comparatively recent.  The smartphone has rendered it obsolete in the space of a mere generation.
  • Finally, as an example of how culture is bending to the ethos of smartphones, a theater in Seattle is planning to allow patrons use their phones during performances.  No more requests that all phones and recording devices be turned off.  As the theater’s executive director said, “Simply forbidding it and embarrassing people is not the way to go.

So we are wiring the building in anticipation of finding ways to make it work over time."

We all know that the march of mobile communications will change fundamentally how we conduct research, but these stories suggest how significant that change can be.

  • Many of us can be quite enthusiastic about the ability of consumers to send their “answers” to us wherever they are.  We can eavesdrop on their reactions in the moment.  But, just like the soldiers in the army’s experiment, they will want to see the hierarchy flatten; they will expect more control.  Consumers will demand to frame the questions as well as provide the responses.
  • The smartphone may have rendered the camera and the watch obsolete.  For many people, it will render the computer obsolete.  Reflect on how quickly netbooks faded from the scene.  We already had a much better “small computer,” our phones.  There was a time when any project began with the question, “Should we do it on-line or over the phone?”  Now we are asking, “Should we have a mobile component in the project?”  In a few years, we will simply take for granted that our consumers are answering on their mobile devices.
  • We will need to engage consumers in a way that makes sense in the culture of the smartphone and the social network.  No longer can we expect to bend them to our style and approach.  Consumers accepted the “social science survey” and the “focus group” when the one-way interview defined the environment.  But, when environment is structured by the shout-out — “Who knows a great place for dinner and meet me there?” — a semantic differential just won’t cut it.

To be successful in the future, to be able to hear the voices of consumers in all their tones and timbres, we will need to adjust our ears to the new reality.  In fact, we are likely to become more listeners and collaborators and less askers and analyzers.  As the Seattle theater director said, we have to find “ways to make it work.”

Tags: mobile research, Market Research, market research tools

The Market Research Event: Understanding Your Consumers

Posted on Tue, Nov 22, 2011

By Darren Breese, Research Director

With all the talk of new technologies and the like in the air, we may overlook the basics.  Yet, one of the themes running through the recent Market Research Event was the notion of simply empathizing with consumers.  This is nothing new of course; it is the core of what we do every day — understand consumers, bring them to life, connect them to marketers.  Every day we put ourselves in the consumer’s shoes, or in some cases actually watching them put on their shoes. 

Sometimes it is hard for clients to truly empathize with their consumers, because quite often they aren’t in the same boat.  They may be more affluent, live different lifestyles, and have upbrings and life experiences that are poles apart.  Despite all of the differences — perhaps because of them — it is the researcher’s job to do as much as to connect marketers and their consumers, and to do this in a way that makes the experience as engaging as possible. Consumer Behavior

A technique used successfully by one researcher at the conference was to force marketers to consume as their consumer does. 

  • Shop on a strict budget (like many of their customers).
  • Shop with children in tow, even if that means “borrowing” kids for a day.
  • Immerse  marketers with triads of like-minded consumers
  • Engage in other non-shopping activities common to the target consumers.
  • And, of course, keep journals to drive their immersion home.

We know and do Immersion extremely well, but Immersion research only works as well as the client wants it to, so we have to constantly look for way to keep things fresh and fun. 

Another way insight managers are using empathy is bringing together cross-functional teams.  We all know how different right- and left-brained individuals think and process information.  It can be extremely difficult for them all to difficult to work on the same page.  By placing cross-functional teams together in the same room with consumers, and holding immersion sessions that help each team member empathize with their consumers, an insights manager got his team to think similarly—like their consumer.  He was then able to hold Ideation sessions that led to productive concept development work. 

In other words, walking in someone else’s shoes has the added benefit of forcing marketers to take of their own.

As we strive to provide marketers with actionable insights and help them connect with their consumers, we must also be consistently looking for new and innovative ways to help them foster empathy for their consumers.  Empathy makes insights real.

Tags: Shopper Insights, Market Research, Market Research Conference

The Multitasking Generations: 13-34 Year Olds Average 4-5 Other Activities While Watching TV

Posted on Fri, Nov 18, 2011
     

New Study Examines the TV Consumption Habits of Generations X, Y and i

Adults 18-24 and 25-34 Most Likely to Connect Social Media to TV Viewing; Teens Most Likely to Watch with Friends and Family

(National Harbor, MD—November 9, 2011) – A new study released today, “Watching Gens X, Y & i,” paints a detailed portrait of 13-34 year old consumers and how they watch television: often while taking part in up to four or five activities all at the same time, from eating, cooking and cleaning to texting, surfing the web, emailing, playing games or listening to music.

“Many 13-34 year olds are multi-media multitaskers, but their social media activities vary depending on age group,” said Char Beales, president and CEO, Cable & Telecommunications Association for Marketing (CTAM), and head of the organization that commissioned the study.

Younger generations have been raised in an entertainment world where content is available anytime, anywhere and on numerous platforms. This study exposes what teens and young adults are watching, with whom they’re watching, where, how often and on what devices.

Although about half of 18-24 and 25-34 year olds follow or “like” TV networks/shows, only 38% of those 13-17 do. The leading social networking activities while watching TV are looking up info (31% of 13-34 year olds), discussing shows online (29%), posting updates/tweeting (24%) and visiting a network or show page (22%). However, these activities are almost twice as likely to be conducted among 18-24 and 25-34 year olds compared to teens.

Click Here To View the Complete Article.

This research, conducted by C+R Research, was commissioned by the Cable & Telecommunications Association for Marketing (CTAM) to investigate the effect of lifestyles and life stages on media and technology usage of younger consumers. It included both qualitative and quantitative online phases in the summer of 2011, and also utilizes data from C+R’s comprehensive syndicated YouthBeat study to provide additional context. 2,124 total interviews were conducted as part of the quantitative phase.

Tags: Market Research Reporting, youth and family research, Market Research, C+R News

Market Research Event Recap: Categorization is Communication

Posted on Wed, Nov 16, 2011

By Hillary Stifler, Director

At the Market Research Event last week, one theme was played out in several presentations – categorization.  And it hit home, as I’m currently working on a study whose goal is to categorize over 100 products in a way that makes sense to consumers!  

Categorization was a major theme discussed by Sheena Iyengar, who spoke about “The Art of Choosing.” People make thousands of choices each day and, as she puts it, face “choice overload.”  She offers three solutions:  The Art of Choosing

  • Cut duplicates and indistinguishable.
  • Categorize the options.
  • Condition the chooser for complexity by offering the easier choices first before working into the more complex choices.  

This advice is not only great for product offerings at shelf, but it is also great for business communications.

Ruben Alcaraz from Meijer spoke on data visualization and gave some great advice that I think ties into the power of categorization. He said, “It’s not that people don’t get it, it’s just that we [market researchers/those sharing our data] aren’t good at communicating it.”  

Really, categorization is communication.  A jumble of data on a page does not tell a story. Humans are visually-oriented and, to be an effective communicator in the visual realm, we must categorize our information in a way that makes sense to our audience.

So, Ms. Iyengar’s three pieces of advice also apply to reporting and data visualization:

  • Don’t show duplicate data.
  • Section off reports (or even parts of a slide) in a way that makes sense to the audience and supports your story.
  • Start with the obvious, more general information and work your way into the deeper, newer information. 

Categorization is not a novel concept; we have grouped and framed information forever. However, I think it is a good reminder that information is far less powerful when it’s not organized in a way that speaks to the audience. And, when organized in a meaningful way, it helps people choose where to focus their attention and it helps the author to tell the story efficiently and with ambiguity.

Tags: Market Research, Market Research Conference, C+R News

“Innovate or Die” Stories about Market Research in an Age of Winner Take All

Posted on Thu, Nov 3, 2011

By Walt Dickie, Executive Vice President

When my two college-age sons were younger they each went through a period of fascination with what I think of as “alternate reality” or “science fiction presented as fact” cable TV programs. Our house was awash for years in aliens hidden in secret government facilities, Sasquatch sightings, and paranormal activities of all kinds.

None of this bothered me too much. Although the sci-fi-as-sci-fact genre hadn’t been so popular when I was a kid, I had loved Erich von Dänigan, and devoured everything I could find about Bigfoot – once even giving serious thought (for a week or two) to writing my Ph.D. dissertation about Bigfoot lore before coming to earth on the realization that my paltry linguistic skills weren’t up to the challenge of the Northwest Coast languages.

But my kids were getting all this via cable at a much younger age than I was, and they didn’t have the grasp on the “science” part yet. I had to have some way of talking to them about what they were seeing that admitted its attraction while warning them about taking it all too literally.

Talking to my older son one day about extraterrestrial visitors, and struggling with the knowledge that the stories of little green men that he found so fascinating certainly had some scientific plausibility, I told him that I thought it would be so cool if it was true. And I saw that what I was trying to say had finally clicked for him. Of course he was fascinated, of course people wrote about this, went to conventions, and searched the skies (and their backyards) for aliens because it would be so cool if it was true. Once the aliens, Yetis, time travel, ghost sightings, and all the rest were seen as powerfully compelling stories, he got the point I had been trying to make about his fascination with them and my parental concern.

That experience had a real effect on me; it was the first time that I fully realized how our entertainment-driven culture has become fascinated by things that are cool to think about, even when we know at some level that they aren’t true. It’s not that we value illusions of truth, it’s that we value cool so highly. This is where internet memes are born – internet flash mobs playing all the variations on a theme that is experiencing its moment of coolness.Market Research Innovations

I think about that experience a lot these days; only I’m not thinking about possible remnant populations of otherwise extinct giant apes wandering around barefoot in the Himalayas. I’m thinking about the unceasing drumbeat of blog posts and articles hawking the embrace of all-out innovation as the only means of escape from certain commercial death.

“Innovate or die” has been with us since the tech boom of the 90s, and it has evolved from a taunt thrown out by a few hard-charging internet entrepreneurs to well-nigh the accepted gospel of modern business. It seems to be especially popular among bloggers and other bystanders; a hair trigger response to any commercial stumble, good for a quick post requiring little thought. Lately it’s become the war cry of the marketing research commentariat, who would have us toss off the shackles of hide bound, obsolete, research techniques – apparently abandoning clients and revenue streams in our haste to get out of long-standing but soon extinct business lines.

C+R and I both survived the 90s – by innovating – and I’ve had a lot of time to think about the events of those years and what they mean to our future. Our experience put me firmly in the pro-innovation camp, and I thought I was comfortable there. Then, a few years ago, my college alumni magazine, Technology Review, which has become a fairly successful commercial enterprise itself, announced that “innovation” would henceforth its theme, its be-all and end-all.

I found that strangely troubling. I’m an MIT grad, but I am not now nor have I ever been an engineer, scientist, or even a technologist in any very serious sense. I’m an anthropologist by professional training, and what technical expertise I have has always been harnessed to business issues.

But when Technology Review started to really harp on “innovation,” I found myself wondering about all of the engineers who had spent their careers improving processes, increasing the accuracy and speed of measurements, making things more efficient, safer, or economical. Certainly those achievements required something reasonably called “innovation,” but that kind of innovation wasn’t what the “innovate or die” crowd were talking about.

You don’t “innovate or die” just by improving something. You invent something “revolutionary.” You “change the world.” You overthrow the old and embrace the new. You cause a “paradigm shift.”

I have a visceral understanding of the allure of paradigm shifts. The voice of Thomas Kuhn, extolling the concept of “paradigm shifts” in the history of science during the 60s, sang as sweet a song in the corridors of academe as The Beatles or The Dead. We wanted to get high, get laid, and cause a paradigm shift that would overthrow the stodgy dogma of whatever we were majoring in – not necessarily in that order. I remember it well. And, like most young academics, I loved the smell of it in the morning.

So why, I found myself asking, does it bother me, 40 or 50 years later, that my alumni magazine has gone all innovation on me? Was I now backing the fuddy duddies? This question really bothered me, raising the possibility, as it did, that I was not only backing them but also joining them.

In the past few years I think I’ve finally figured it out. Kuhn was writing about big-S Science, and the Tech Review has chosen to concentrate more on Big-T Technology rather than the small-t technologists it used to write about. Paradigms are the right thing to think about when you’re talking about big-S, big-T stuff. I thought so in the 70s, and I think so now.

But if you’re a small-letter scientist, technologist, or marketing researcher, then the issue isn’t as clear. Should you, in fact, stop wasting your time on something that will be displaced by a paradigm shift?

Let’s imagine you live near Lake Michigan, as I do. It’s summer, it’s warm, and the sun is shining. The gorgeous blue waters and fresh breezes beckon. You know, of course, that the paradigm of gorgeous weather will shift, possibly very soon. And, this summer at least, you have every reason to believe that 60 mph winds and 20’ waves may make that shift very unpleasant indeed. For purposes of this little example, you have no access to any weather forecast of any kind. What do you do? Stay on shore or get in a boat?

The fact is that, until the advent of modern computerized weather models, no forecasting tools could consistently beat the prediction that tomorrow’s weather will be very much like today’s. Your best bet, then, in the absence of better information like an online satellite view, is to predict a continuation of more of the same. Shove off, hoist the sails, and enjoy the continuing lovely weather. At least in the short run.

Let’s try another analogy. You’re a marketing research company that has a good-sized survey research business. You’ve been doing well on survey research for many years, you negotiated the transition from phone and mall to the internet, and you’re doing well getting your head and hands around the mobile revolution.

But you see a paradigm shift coming. Big data could shift the paradigm and undermine the market for survey research because, given enough data, what people actually do as reflected in their searching, shopping, purchasing, friending, tweeting, and other choices on millions of web sites is a much more reliable indicator of their future behavior and their opinions. And a paradigm shift in cognitive science says that people don’t even know what they’re thinking when they’re thinking it; that what they think they think and whatever “thought” it is that drives action in their world are two totally different things, often in conflict with one another. Who will care about what people imagine they think in the future? Your survey research could be shifted right into the trash.

Or will your recent good results from survey research continue, like the weather, at least in the short run? For that matter, even if your business tanks in the long run, what exactly does “long” mean? Do you get out of the business today? Next year? Suddenly? Gradually? And what does “out” mean? Quit? Put less emphasis on selling and marketing? Stop developing completely and freeze everything just as it is today?

Since we’re talking here about you, a small-m marketing researcher, not the big-M marketing research industry as a whole, it’s pretty hard to know the right answer. Certainly these, and other equally large, evident paradigm shifts, are going to impact Marketing Research – or Consumer Insights, Decision Guidance, or whatever we wind up calling what we do. But what happens at big-M scale may not be what happens in your immediate neighborhood.

Famously, buggy whips went out a century or more ago, killed by the big paradigm shift to internal combustion engines, automobiles, and global warming. But there are still buggy whips being made and sold, and someone is probably making a nice living off them. Along about the same time that the horse-and-buggy went out, a huge industry based on fancy feathers for ladies’ hats, that had shipped tons upon tons of feathers annually, died out, swamped by a sea change in fashion. But there are still feather merchants to be found, and their industry, once dominated by sales to fly fishermen and Vegas-style costumers, is booming thanks to a fad for hairdos incorporating feathers. I remember reading a story a few years ago about a local firm’s turnaround – they had gotten out of their struggling “modern” electronics business and established themselves as the dominant player in the global market for replacement vacuum tubes for legacy equipment and were making record profits! Vacuum tubes! Who knew? And closer to home, there are still many phone rooms plying their trade for marketing research long after the online paradigm shift moved the industry in a new direction.

Let me be clear: I’m not arguing that it’s better, wiser, or more lucrative for your career or business to live on the backside of a paradigm shift, only that there are a lot more choices and possibilities than just “get with the new paradigm” or “drop dead.” Anyone who tells you otherwise has either never managed a real business or going for sensationalism over sense. Hooey!

It’s pretty clear why we’re so enamored of “innovate or die”– you don’t have to have paid much attention in any of your lib arts courses to recognize and understand the appeal of a story arc. We humans all crave stories and are fascinated by them, so condensing the complexities of actual life into the simple narrative of “innovate or die” is perfectly understandable, even if pretty much useless as advice. Innovators on one side – successful, profitable, with it, and future-oriented – and dead companies – foggies, fools, and bankrupt stick-in-the muds – on the other!

It is, I think, a winner take all fable for a winner take all age.  It would just be so cool if it was true!

Tags: Market Research, market research tools, Misc

Market Research Use of Word Clouds vs. Data Journalism

Posted on Thu, Oct 20, 2011

By Robert Relihan, Senior Vice President

Those of us who face the daunting task of sifting through mounds of verbal data — forty individual interviews or a three week on-line community — synthesizing all of those words, and presenting our analysis clearly, succinctly, AND with impact have been intrigued by word clouds.  They seem simple and elegant; they reduce all of those words to a picture of the important themes.  Now, there are numerous web sites to help us create word clouds, Wordle being only the best known.

So, why do I feel vaguely dissatisfied when I look at word clouds, even the ones I have created?  They often seem to miss the point or be overly simplistic.  Creating a word cloud sometimes feels like I have cheated myself and my audience.Market Research Wordle

Jacob Harris of The New York Times makes the case against word clouds and, in the process, gives a brief primer on how to report its data.  It is an incredibly useful article.

He begins with a concept that should become a mantra to market research professionals.  The critique of data clouds is based on the principles of data journalism.  When many of us began our careers, the model for reporting was the academic paper.  I wrote reports a long time ago with footnotes.  Now we strive for clarity and simplicity.  The magazine or newspaper (on-line versions, of course) is our guide.  Data journalism should be our art.

“Visualization is reporting, with many of the same elements that would make a traditional story effective: a narrative that pares away extraneous information to find a story in the data; context to help the reader understand the basics of the subject; interviewing the data to find its flaws and be sure of our conclusions.  Prettiness is a bonus; if it obliterates the ability to read the story of the visualization, it’s not worth adding some wild new style or strange interface.”

The ways Harris point out how word clouds go wrong provide us with a road map for good reporting or, rather, good data journalism.

  • Word clouds are based on a very rudimentary textual analysis.  In most cases, a phrase-level or a thematic analysis would provide richer and more penetrating analysis.  The general lesson of this observation is that we need to focus on the concepts that knit the words together consumers use and not on the words.
  • Word clouds are often used when textual analysis is not the appropriate tool.  As Harris says, in our analysis we should not confuse “signifiers with what they signify.  We need to use the appropriate methods for getting below the surface of consumers’ comment.  Simply digesting their words will not do that.
  • Word clouds have a dirty secret.  They really aren’t analysis.  They leave readers with the task of peering at the image and discerning the meaning themselves.  Word clouds make the assumption that the meaning is obvious.  But, any analysis worth its salt requires some explanation; it requires framing and focus.
  • Finally word clouds miss the narrative.  I am not saying what we write or present should be long, dreary marches through the data.  Hardly.  We need to find the thread or threads that bring fresh insight to a particular area of consumer behavior.  There are often several reasons why two words might dominate a word cloud.  We need to create the story for the reader or listener that that makes just one of these reasons the most compelling and the most relevant.  And, incidentally, that narrative still can be visual.

A word cloud tries to make us believe its immediacy produces insight when, in fact, it may mask the narrative we “data journalists” should be creating.

Tags: Market Research Reporting, Market Research, market research tools, Word Clouds

5 Ways to Know If You Have an Insight or Observation

Posted on Mon, Oct 10, 2011

By Robert Relihan, Senior Vice President

For as long as the phrase “Consumer Insight” has been used, I am amazed that I still see discussions of its meaning.  Don’t we all know what an “insight” is by now?

Well, maybe not.  The word certainly has been overused; every observation about consumer behavior and attitudes becomes an "insight."

Market ResearchAnd, unlike a mere finding, it has an air of inevitability, not to be questioned.  Perhaps, the use of "insight" is an example of language creep — using words of greater and greater intensity to describe our actions in an attempt to endow them with more importance than they deserve.  It is a bit like using wind chill factors to convince ourselves that winters are re-e-e-ally cold now.

Yet, the word is useful.  Who doesn’t want to get into the head of a consumer?  That’s what an insight is — an authentic vision of how consumers view themselves and connect with brands or categories.

So, here’s five ways to know you have a true insight and not a mere observation.

  • An insight is clear and simple.  One short, declarative sentence is best.  If you need three sentences to explain it, you don’t have an insight.   Most of all, it can’t be a multi-layered, logical construct.  A syllogism is not an insight.
  • An insight is a surprise. You may discover insights, but you can’t necessarily search for them.  Insights into the consumer give us new and fresh perspectives; they are unexpected.  Consequently, we are more likely to discover insights by being open and not wedded to particular methods.
  • An insight is a game changer.  If an insight is a surprise, if it gives us an unexpected vision, it must drive the development of different products and different ways of communicating them.  The recognition that consumers wanted small indulgences in every category transformed the coffee shop into Starbucks.
  • An insight is often a mash up.  How can you be surprised?  How can you be open to discovery?  If there is one rule to developing insights (I resist calling it a method), it is that you need to combine perspectives.  You might never have had that “Starbuck moment” watching people getting coffee in traditional coffee shops.  But, if you also observed them in bakeries and wine bars, the light would go off…insight.
  • An insight is not immediately translatable.  We have spent most of our careers looking for “actionable results.”  It is a laudable goal and, in many circumstances, it is an essential goal.  What good would a taste test be without actionable findings?  But, if insights are surprising, game changers, they may not have a direct immediate utility.  We need to be prepared to follow the insight, to push the insight to its ultimate game-changing conclusion.

Check us out at C+R and discover how we can be your partner on that journey of discovery, from insight to game-changing conclusion.

Tags: Market Research Reporting, Market Research, Misc

Why Marketing Research Gets Mobile Wrong

Posted on Thu, Sep 29, 2011

By Walt Dickie, Executive Vice President

Mobile. The next big thing, right? Well, maybe or maybe not. I’ve just been Googling mobile marketing research topics, downing a few white papers, and reading some conference presentations, and it strikes me that an awful lot of marketing researchers simply don’t understand what mobile means for the research business or the impact it’s going to have.

Here’s the problem: Imagine you’re a buyer of marketing research. Today. Now. You’ve got more questions and issues than you can easily cram into your limited budget and timeline. You’ve worked up a survey research design, and you’re balancing sample costs against the sample sizes you need for the analysis.Mobile Research

Then someone points out that some of your key segments are going to be pretty difficult to get. People who are very mobile. People who don’t have home computers and internet connections. “But they have cell phones,” says someone else. “We could maybe interview them on their phones!” Briefly, your spirits soar – you’ll add an online sub-sample!

Then the realities hit: short interviews, something called a “mobile template,” increased sample costs, a whole new technology to tackle, and a methodological nightmare to merge the mobile data and findings with the rest of the project. The benefits aren’t enough to justify the approach. “Great idea, though. We’ll have to keep that in mind.”

This is totally wrong. It’s not just wrong today, for today’s project – you almost certainly made the right choice for your already over-burdened, under-funded current project. But it’s wrong for tomorrow, and wrong for the industry.

Because mobile isn’t simply a matter of sample. Mobile is about culture. More specifically, it’s about culture change. And you’re very likely to be part of that change already.

Don’t believe me? Do you have a smartphone? Have you had it for at least six months? OK – answer this question: Out of the last ten times you checked your email, how many times did you use your iPhone, Blackberry, or Android phone? Office computer? Home computer? Tablet?

I’ve been paying attention to my own behavior for the last week or so, and I’d guess that out of ten tries, I check email 4 times on my phone, 5 times on my office computer, once on my tablet, and I probably didn’t use my home computer at all.

Now think about those times when you used your phone: Where were you? For me, I check my email using my phone from lots of places: sitting at home with the phone at my side, driving to or from work, running errands on weekends, in the office when I’m away from my desk. And that’s where the culture shift lies.

I have my phone on me most of the time – it’s sitting on my desk as I write this on my office computer. And I use it almost everywhere, including a few times when I’ve already got Outlook running on a computer right in front of me. I’ll bet you do the same.

My phone is with me most of the time. It’s my personal device, the one I have close to me no matter where I am or what I’m doing.

The smartphone is the logical culmination of what people talked about in the 80s and 90s when they got enthused about how personal the desktop computer was. Here, for the first time, were computers designed to be used by a single person, with all kinds of ways that they could be customized to that person’s tastes. Screensavers, wallpaper, custom menus and button bars, programs you liked and used – all eons away from the big iron where you might have had a timesharing account.

The smartphone is the most personal PC yet. The longer you have one, the more you find yourself using it, often for things you thought you’d never do on such a small device, in places where you never used your previous “not”-smart phone. Over time, you find yourself expecting to use it for just about anything that you can use it for. And you’re increasingly annoyed when you can’t – when a web site doesn’t render using a mobile template and forces you to endure a nightmare of zooming and hitting links inadvertently because it’s cramped, crowded layout is hostile to your little screen and big fingers. Or when a survey invitation dumps you into a questionnaire clearly designed with the expectation that you’re sitting at a keyboard, looking at a big screen, and navigating with a mouse. When you’re not in the “mobile sub-sample” because the marketing research industry didn’t get how the culture changed when smartphones appeared in force and everyone used them for everything. Because MR got mobile wrong.

Tags: mobile research, market research tools, C+R News