Makeover Challenge

Winning Isn't Everything

Posted: September 21, 2009 by
Ron Romanik

In typical focus group settings, participants want to help you. They want to sound smart. They will give you answers when they don't know the answer, and almost always on the most superficial aspects of your product. Participants will want to try and "fix" your product. And if they're seeing multiple design options, they are going to choose their favorite. And they're going to call it "the winner."

No matter what the product, package, or setting, there's always going to be a chosen winner. However, knowing what focus groups choose as a winner does not always help the design process. The problem with focus groups, and many kinds of research, is that they are unable to capture the essence of the first impression.

Taking another tack

Not to downplay the importance of focus groups for research, but perhaps they are not always the best form of research when it comes to package design. Considering this, the William Fox Munroe design firm created a research process that used richer diagnostics, eliminated proactive consumer analysis, and measured design only against predetermined marketing objectives.

Called Design Checkā„¢, the process is an online research tool that combines monadic methods with quantitative research. Monadic meaning each respondent sees only one of the designs being tested. This forces research subjects to evaluate one particular design, quickly, without being influenced by how other designs look.

William Fox Munroe applied its Design Check research tool to one package, the Carpet & Room Odor Eliminator, in the 2007 Package Design Makeover Challenge. Double-opt-in respondents were sent an email solicitation asking them to take a brief survey and earn a chance to win a prize in a sweepstakes. If interested, the respondents clicked on a link that transported them to the survey site where they were asked demographic information on gender, age, and residency.

The population under study consisted of a total of 1,000 respondents. 500 viewed the old design, and 500 viewed the new one. Questions were randomly rotated to prevent order bias. With this sample size, the data has a 4% margin of error.

Respondents were asked to view one design being tested. After 10 seconds, the design disappeared and the respondents were asked to rank the image they had just seen against a set of predetermined measurements that included product attributes, emotional responses, and behavioral responses.

William Fox Munroe uses a benchmark of 70% (Strongly Agree plus Somewhat Agree) to predict the market success of package designs. If scores on key attributes are over 70%, it is likely that the product will have success in the marketplace.

Is contemporary better?

In a nutshell, the winning Tridimage design of the Carpet & Room Odor Eliminator did not score significantly better than the original overall. And that's okay; it happens. Clients always want to see scores for the new design jump off the chart in order to claim a victory.

In this case, the original design did score very well against a number of key attributes. As long as it doesn't significantly outscore the redesign, and as long as results do not indicate consumers would be confused by the launch of the redesign, then there really is not a problem.

While consumers agreed the new package design was more contemporary, both designs scored evenly that this was a premium product, a product that was easy to use, and a product that would make their homes smell clean and fresh.

One or two messages may have become lost with the new bottle and label redesign. The 100% natural claim was one of them, as 65% agreed the original design was 100% natural, while only 55% agreed against the redesign. Not surprisingly, scores against the redesign for "Works to control household pet odors" fell significantly, because that message was dropped. And while 10% more agreed the old design used the power of real citrus, both designs scored above the 70% approval rating.

Purchase intent scores were notably high, and interestingly, even with each other across the two designs. A high percentage (86%) agreed that if they were shopping for this type of product, either Citrus Magic design would catch their eye on shelf. And 80% committed to wanting to try the product regardless of the design.

A radical redesign like this Citrus Magic case study can be risky as it may confuse consumers, adversely affect market share, or lose brand equity. But the William Fox Munroe research does not necessarily tell us that this would be the case, and Design Check is just one tool for insight when evaluating package designs. While the redesign does not show more appeal than the original, and some of the original benefit messaging has been lost, scores against a number of key attributes and purchase intent indicate to move forward with the updated look.

To find out more about Design Check, William Fox Munroe's proprietary online packaging research tool, visit