Saturday, July 19, 2008

Measuring Engagement and Marketing ROI, Part 1

Kim, a reader of this blog, shared some comments about marketing ROI and got me to thinking about this complex and challenging topic. I blame her for this lengthy but hopefully thought-provoking overview of marketers' quest to quantify engagement and ROI. Your input on this two-part blog post would be greatly appreciated!

There's a lot of buzz in marketing about the challenge of measuring consumer engagement and ascertaining ROI, and for good reason. Developing and maintaining more engaging consumer experiences such as live events, advergames, social networks, and branded content can require significant investments. Marketers need to know these investments are worthwhile in relation to alternative programs.

The problem is how to measure and compare engagement across multiple channels. This is particularly important for experiential marketing programs because the number of people "touched" is much less than for mass media programs (but the touch is far more substantial and meaningful). If a marketer must decide between spending $50,000 on a full-page ad in a national publication with 5 million subscribers or spending the same $50,000 on an event that will attract 5,000 prospects, you can bet there will be a great deal of discussion about the deeper engagement of the live event and what that is worth to the brand.

Those in Experiential Marketing--particularly the Interactive side--have a tendency to gripe that they're held to a different standard than those who execute traditional mass media campaigns. Marketers never inquire as to the ROI of a specific billboard or pore over reports on the effectiveness of a print ad on a monthly basis, but they will will do so for banner ads, online media buys, microsites, PPC programs, and social media programs.

These gripes aren't completely without merit, but those complaining are seeing the glass half empty. While the measurability of experiential programs does come with a burden to do the measurement, prove ROI, and constantly improve the program, it also provides the opportunity to do the measurement, prove ROI, and constantly improve the program.

The challenge we face is to use all of the monitoring and analytical power at our disposal to validate the effectiveness of our marketing strategies and tactics. But with the plethora of metrics available to us, which ones permit us to measure engagement? Which metrics are best to validate ROI? The answer is all of them and none of them.

Here are some categories of metrics along with their strengths and weaknesses for measuring engagement in a meaningful way:

Exposure Tallies: The easiest and cheapest type of marketing assessment is to simply count the consumers who are exposed to the marketing program. We can count magazine subscribers who see a print ad, viewers who see a TV ad, people who walk past a mobile event footprint, or Web surfers who view a page containing a particular banner ad.

Problem is, exposure counts are a measure of reach and not a gauge of engagement. Using exposure tallies allows us to know how many people theoretically viewed a print or banner ad, but we cannot measure the impact of this ad exposure without gathering additional data.

One way Exposure Tallies are used is to compute a "Cost per Touch" (CPT), calculated by dividing a program's financial cost by the number of people exposed to the program. Since the depth and impact of the exposure isn't considered, this calculation is almost meaningless; worse yet, CPT turns marketing metrics into a game.

For example, some event agencies will quantify as a benefit of a mobile program that thousands of consumers will see the brand's logo on the side of a truck while the event is in transit. We'd all agree that seeing a truck on the highway imparts minuscule brand benefit at best, but including these exposures decreases the CPT and makes an event program seem more attractive. (If seeing a logo on the side of a truck really created significant brand engagement, wouldn't U-Haul be the most valuable brand in America?)

Action Tallies: Exposure Tallies do not consider occurrences such as consumers walking into an event footprint or visiting a site. These represent actions taken by a consumer; in order to experience an event or visit a site, the consumer must have been sufficiently engaged to decide to enter the event or click a link. As a result, Action Tallies--counting the number of desirable actions taken by consumers--are a much better way to assess engagement.

For example, the easiest way to capture the engagement of an online ad is the click rate (the ratio between the number of times consumers click on an ad and the number of times they see it.) Since a substantial level of attention and motivation must be attained before a consumer clicks on an ad, this seems like a relevant engagement metric; and it is, but only under certain circumstances.

A banner ad that promises deep discounts will get clicked often but produce little brand engagement while an ad targeting the right consumer with the right message may see fewer clicks but furnish better and more engaged consumers. This may be an exaggerated example, but it does demonstrate that an action isn't an action isn't an action. Action Tallies provide no basis for comparing the value to the brand of different actions across various media.

Mass media advertisers have been attempting to increase accountability by building trackable actions into ads. While this is a very welcome development toward improving the trackability of traditional media, this tactic isn't widespread and would likely lose its power if overused. An example of this type of campaign is the airport out-of-home campaign for BMW: The brand used airport signage containing SMS short codes at major airports nationwide. A consumer who sent a keywords to the short code received a deep link to a mobile application promoting BMW’s new convertible, and BMW could track which airport signs were best at driving interest based on the submitted keyword.

Another valuable purpose for Action Tallies is to measure improvement over time. For instance, if a live event is failing to draw consumers into the footprint, counting event attendees as different traffic-driving tactics are tested can be helpful to improve the flow of interested prospects. Of course, what you do with those people once they are attracted to your event is where the real engagement lies, and this is why most Action Count metrics fall short of furnishing marketers with objective engagement or ROI results.

Transaction Metrics: Transactions are the actions that matter. They involve consumers exchanging dollars or information with the brand.

Counting the number of purchases or adding up the total monetary value of transactions generated by a marketing program is a good measure of engagement. This works especially well online and overcomes the click rate biases described previously. Banner A may produce 1,000 clicks and 10 purchases while Banner B may produce 500 clicks and 25 purchases; in this case, Banner B is obviously creating a greater level of engagement and ROI, despite furnishing fewer clicks.

Another typical form of transaction measurement is prospect acquisition--securing new consumer records via online registration, business reply cards, sweepstakes entries, and similar means. Consumers who are appropriately engaged will be enticed to furnish their data; ones that aren't will withhold their personally-identifiable data.

Transaction Metrics can provide insight into consumer engagement that directly produces monetary value, but only for programs with a direct response objective. This makes Transaction Metrics perfect for some channels (PPC ads, banner ads, direct mail) but unavailable for others. For instance, how can a CPG brand site measure engagement when the goal isn't to motivate a purchase but to inform consumers, enhance brand consideration, and improve purchase intent?

For my answer to that question, please return to Experience: The Blog tomorrow for the second part of this article on engagement and marketing ROI. We'll explore Content Engagement, Blended Metrics, Brand Lift Measurement, and why ROI is so darn elusive in the marketing world.


Anonymous said...

Hi Augie,

This is great, you make excellent points here and I'd be happy to take a tiny part of the blame for this terrific post. (You take the rest ha!)

Most marketers are definitely heading towards somewhere in between Exposure tallies versus Action tallies versus Transaction metrics - I think they need all three, integrated as a team.

It's similar to how only a few years ago, you'd hear how Direct Marketing and Brand Marketing were completely different breeds of marketing and required opposite personalities - but interestingly enough, recently you hear a lot about how they have come to start overlapping. The Direct Marketer can no longer ignore emotional value and personal "touch" points. The Brand Marketer can come to appreciate some of the numbers involved with DM.

I like your examples on some of the tactics companies have utilized. Looking forward to part two!


Augie Ray said...

Thanks Kim! Appreciate the positive comments!