Masso Group

Consumer surveys remain a critical tool to diagnose brand deficiencies or growth opportunities. A restaurant chain's decision to spend on better staff training or new décor might hinge on consumer feedback. Similarly, whether a brand positions a product on a functional or emotional benefit might be guided by consumer reactions to each appeal. Unfortunately, surveys rarely, if ever, provide a direct window into consumers' minds. A host of biases exist that can lead to errors in measurement or inaccurate responses. Yet marketing executives rest major brand decisions on such data. That makes reducing survey bias crucial. The notion that surveys aren't perfect

 is hardly new. Marketers easily recognize errors caused by imprecisely worded questions or poor consumer attention, and they have many ways of mitigating them.

But miscommunication is not the only source of survey bias. In a recent paper appearing in the Journal of Marketing Research, even when a question's meaning is transparent to consumers, another insidious bias can emerge: response substitution. We define this as a tendency for consumers to provide an answer to a question that reflects attitudes or beliefs that they want to convey, but that they have not been asked about.

Consider, for example, a voter who supports President Obama but is indifferent about the health-care reform package he backs. If the voter is asked about his support of health-care reform, the voter might report a favorable attitude in order to express his support of the president. There is no misunderstanding of the question being asked; rather, the respondent uses the question to express an opinion that was not asked about.

To test if response substitution occurs, in one of our experiments, participants read a scenario that described an individual acting in a wasteful manner (for example, buying an expensive fondue set for a onetime use). Participants were asked how wasteful and intelligent the individual was with the order of these two items counterbalanced and appearing on separate pages. In general, participants viewed the person as wasteful. However, participants reported the individual was less intelligent when they had not first been asked about the individual's wastefulness. Thus, when it was unclear whether participants would be able to express their negative opinion of the individual's wastefulness, they used the question about intelligence to express this negative opinion.

Why does response substitution occur? The answer lies in consumers' deep-seated need for self-expression. The need to express one's opinion is powerful and exists whether people are asked for their opinions or not, particularly when individuals view their opinions as important. To illustrate, in another experiment we asked participants to read about a candy-bar company associated with either moral or immoral behavior. When asked about the product's quality, individuals were more likely to judge the candy bar as high quality when the company was moral versus immoral. However, this effect was much more pronounced among participants who indicated the company's moral behavior was important to them.

An additional experiment found the effect was strongest when people believed the question they were asked could be used as a means of successfully conveying their general dissatisfaction. If consumers did not believe their answer could reflect the attitude they wanted to express, we found response substitution was attenuated.

Although response substitution occurred across multiple experiments, there is good news for marketing practitioners. Response substitution can be significantly alleviated by alerting consumers that they will be given an opportunity to freely express any opinions they are not asked about. To test this idea, in the study involving the moral or immoral candy-bar company, we told a subset of participants they would have an opportunity to share any additional thoughts about the company at the end of the survey. Among this subset of consumers, the effect of the company's moral behavior on ratings of the candy bar's quality was significantly reduced. That is, having an opportunity to express opinions they were not specifically asked about in an open-ended format reduced consumers' tendency to engage in response substitution.

Why is response substitution such a concern for marketers? Because when consumers are dissatisfied with one aspect of a product or service that is not asked about, they might use other questions in the survey to express their dissatisfaction. To use a recent example, BP's oil spill has likely left a scar on the brand that will not soon be forgotten by consumers. What does this mean for BP? In the future, they should recognize that consumers might have a strong desire to express their disappointment in the oil spill, even when not asked about it. As a solution, to improve the accuracy of future surveys (for example, surveys asking about the quality of BP's service stations), BP could inform consumers that they will have an opportunity to share additional opinions about the company at the end of the survey.

Although most decisions require a cost-benefit analysis by CMOs, in this case, the cost is disproportionately low compared to the potential benefit. If consumers are merely informed they will be able to express their opinions, the risk of response substitution drops -- and brands might get insights into deficits or opportunities they otherwise might have missed. It's a simple change in format that could lead to accurate and diagnostic -- rather than contaminated and superfluous -- feedback.

 

(Source: Advertising Age)