Analytics: Can We All Win?

Tami Dalley

So testing is finally sexy again…finally.

Just like oversized sunglasses and Mickey Rourke, A/B and Multivariate testing have had facelifts (no pun intended) and are back in the game. Seems like those direct marketers knew what they were doing all along, yeah?

The idea is simple: Send your online traffic to different pages, see which one performs better…we all walk away happy. But how do we really measure success? And how can we do it better?

After all, we’ve been hearing it for years: “Select and test just one measure of success.” It makes sense, right? Everything gets muddled if you don’t stick to one goal post. The guys on the Product team will say the clear winner is the page that generates the most downloads, while the Business Development department will argue that the page with the most contact form submissions is better. It seems pretty clear: Pick one measure of success or you’ll go in circles forever.

How can we possibly measure success with multiple goal posts? Well, the answer is simple, and we’ve been doing it for years.

The all-important question is then, “How do we pick just one success metric, and how do we make sure it’s the right one?

But what if we took an entirely differently approach? Why not cross party lines, shake a few hands, pat a few backs and finally agree that we’re all right?

I know, I know. Too idealistic, right? How can we possibly measure success with multiple goal posts? Well, the answer is simple, and we’ve been doing it for years – we just haven’t readily applied it to our online testing tools. The solution? Use an index to measure page testing success.

An index allows us to consider multiple conversion points in combination with each other in a single scenario. This is done by simply giving each conversion point a value in terms of how much it contributes to overall business success.

For example, a download might count as 2 points, while a lead gen counts for 5 points. Winning pages are determined by the highest number of points (analyzing which pages achieved the objectives of multiple success points – rather than determining success on one point to the disregard of all others).

Of course, you’ll need to first engage in frank, insightful discussion about the appropriate “weight” to give each conversion point. You’ll also want to make sure you’ve conducted some fairly robust analysis on how much each conversion point contributes to overall business success. Certain conversion points will likely resonate better with specific customer groups; thus, understanding the language that compels these users to convert can provide targeted insight into their behavior and response patterns, helping place a value on these customer segments.

Once you have this weighting system in place, all that’s left is to inflate the appropriate values in your testing tool. Be sure to use a field that captures figures cumulatively – we often recycle the revenue field for this – so that you can predict a winner based on the page with the highest overall point value.

This scoring approach can also be useful when optimizing paid search or other online marketing campaigns that generate multiple conversions. Every keyword will generate a certain number of points, and looking at the total scores can then help provide insight into the value of each keyword.

It’s one step closer to testing nirvana…

Tami Dalley is the Director of User Experience Optimization at Acronym Media.

Share and Enjoy:
  • Sphinn
  • StumbleUpon
  • Mixx
  • Reddit
  • Digg
  • del.icio.us
  • Yahoo! Buzz
  • Facebook

Join the Conversation:

  • Comment - Share your thoughts!
  • Ask a Question - Check back often for an answer from one of our experts!
  • Suggest a Topic - Let us know what YOU want us to write about!