M3 Message Testing

Until recently, the most important part of communication research was also the weakest link. Namely, it was virtually impossible to do effective, meaningful, and reliable message testing. 

The problem was the telephone interview. Interviewers would read a series of messages and ask respondents to rate each one on a 5 or 10-point scale. Respondents would quickly tire of the process, fall into patterned responses and, typically, rate all messages similarly. The result was a muddle of data with little precision or differentiation and even less strategic value.

As reliable, projectable online research became available Prime Group developed a new methodology called MaxDiff Message Modeling or M3. It is based on the Maximum Difference Scaling platform which had been in use in product and branding research and which we adapted for message testing.

M3 is a “forced choice” methodology. Each respondent sees a series of screens displaying four of the messages being tested and is asked to choose the "best" and "worst" message on each screen (see screen shot below). Each message repeats multiple times throughout the exercise, each time against different competing messages. A sample of n=1,000 produces an amazing 40,000 unique data points yielding extreme precision and differentiation (exactly what was lacking in traditional message testing).

The backend algorithms produce a precise "persuasion score" for each message -- a rank ordering of the effectiveness of the messages and how much more or less effective each message is than every competing message. The software also produces a “reach analysis” showing which messages work best together to reach the largest audience.

The end result is a great deal of confidence that a communications campaign is based on the most compelling messages which, in turn, produces uncommon message discipline within an organization. And that means whatever resources are deployed will have maximum impact with the targeted audiences and will most increase the likelihood of ultimate success.