The ''skinny'' on email marketing statistics

So to bring this year to a close we have decided to end off with a discussion on evaluating campaign results.

I have set out this article as a process of questioning that will deliver information to enable decision-making. As with most business strategy evaluation processes it is a feedback loop that will refine your emarketing control process.

Question 1: What can I measure?

Depending on the sophistication of your emarketing system, you can measure quite a bit.

Typically the following metrics are available:

• Open rates (remember these are only available on html emails, more about this later)
  
• Click-through rates
  • Total - the total number of times a link has been clicked
    
  • Unique - the number of recipients who have clicked on a link
    
  • Viral - click-throughs resulting from forwarded emails
• Bounce rates
  • Hard Bounces - The domain no longer exists on the server, or the email address no longer exists on the server
    
  • Soft bounces - Change of email address, out-of-office replies, etc.
• Subscriptions/Form completions
  
  • Total
    
  • By Date
• And sometimes, Click-stream behaviour
  • Page views
    
  • Number of pages viewed
    
  • Which pages viewed
    
  • Click-to-purchase
    
  • Purchases

Ideally you want all of this information stored against each recipient for each campaign. Depending on your system you may have data like this stored against each recipient (barring viral click-throughs), or only aggregate statistics.

Individual data enables more data mining. For instance, you may want to find out what the purchase rate of over-35's who clicked on "link A". Why do you want to know this? Well, link A may be the perfect step to get under-25's buying your product, but it may be a spanner in the wheels of your sales cycle for over-35's, but you will never know if you don't have individual recipient data.

Question 2: What am I actually measuring?

As mentioned earlier, "opens" are not registered for text emails so depending on how your emarketing system works out open statistics you may need to adjust your statistics to provide a more accurate open rate. Usually, the same open rate is assumed for text email as for html emails because the subject line is the same. This, however, may not be a valid assumption. For instance, higher click-through rates on text mailings may not be an indication that recipients are more receptive to text mails and are more likely to click through to your site, but rather that more text are being opened; perhaps your html emails are being put into your recipients' "Junk" folders.

Further, what are your click-through rates measuring? Are they measuring percentage of click-throughs out of opens or percentage of click-throughs out of delivered? The former measures conversion of opens to click-throughs, the latter conversion of delivered to click-throughs. The standard is click-throughs out of "delivered", due to the html-text open issue mentioned above. You can adjust the figure to work out the conversion of opens to click-throughs.

All of the above rates depend on the delivery rate. The delivery rate should describe the number of delivered emails. It should be equal to emails sent less hard bounces (see above). Not all systems work it out this way; you need to be aware of how your system processes bounces and what this means for your delivery statistics. To evaluate your bounce statistics you will need to follow the same line of questioning.
Question 3: How should I interpret what I am measuring?

One of the key issues in evaluating campaign results and statistics is how much we can describe consumer response in terms of the metrics our emarketing generates. What does an "open", or for that matter a "click-through", mean? Does an "open" mean "I am interested in your offer", or "I open up all my SPAM"? Is a click-through an indication of a good prospect or a casual browser?

Non-electronic direct marketers have this nailed down; they measure "closes" - essentially, answers to questions that indicate an interest in purchasing the product on offer. After a pre-determined number and type of "closes", the customer is ready to purchase and the salesperson pulls out the paperwork and seals the deal. This is a much simpler situation than the emarketer faces, the emarketer is often promoting more than one product and has to make inferences about recipients' online behaviour because she cannot view the recipients' responses.

You need a similar framework.

Depending on your product or service your online/offline sales cycle and "closes" may be slightly different, let me give you a simple example:

Open : Interested in offer
Click-through : Evaluating product/s
View 3+ catalogue pages : Interested in product range
Fill out "Get a salesperson to contact me" form : Further Interest
Accept Sales representative engagement : Serious shopper
Positive sales meeting : "Sell me your product"

Normally these processes are a little more complicated and will not only follow one straight route to a sale, but go through a more roundabout route - in much the same way a salesperson might.

Question 4: What are changes to my communication having?

Are the various processes and content in my sales cycle stimulating my prospect to progress through my sales cycle? Here we want to compare statistics from before a change in communication to after. We want to know, for instance, whether a new format of subject line increased open rates or not. We also want to know whether it increased click-throughs, subscriptions, etc. Further, we may want to know what the ideal discount is to reach our sales target for each target group.

There are two ways to do this. The first is a more simple method and involves altering one aspect of your communication at a time and measuring the results. There are two problems with this:
1. You have to wait a long time to generate a reliable statistic and may have to keep unsuccessful content for a long time
2. You cannot change any other content otherwise you will not be able to separate the effects of the various aspects that are changing

A better method is to use regression analysis to separate the effects of the various changing aspects in your communication. Regression analysis quantifies the relationship between a particular explanatory variable (e.g. discount) and a dependent variable (e.g. open rate, form completion, sales, etc.); it measures the change in the dependent variable for a certain change in one explanatory variable. It allows you to change variables in your content more frequently and still measure their effects, but it does require someone with statistical skills and knowledge to be performed properly.

A very important point to note here is that frequent large-scale changes to your list (particularly regular database purchases and additions) will affect your statistics and make comparison impossible. For instance, the addition of large numbers of recipients with a completely different demographic is an extreme example of this.

Question 5: How useful are my metrics or indicators?

In other words, are my "indicators" (or metrics) a good indication of desired recipients' intentions?

To find this out we need to work out the correlation between the final statistic (Sales, or "rep request" form completion) and any of the other statistics (say open rates, click-through rates, etc.). The higher the correlation, the better the statistic indicates the intention acted upon and measured in the final statistic.

We may also want to examine various steps in the sales cycle. To do this we could run a regression of a pre-step metric on the metric after the step:

[Post-Process1 metric] = a + ß[Pre-Process1 metric] + e

A low or negative ß could indicate that the metric before Step1 is not a good indicator of intention to go to step2.

It could also mean that:

Process1 has a problem with content, ease of use, etc.

Process1 should be eliminated

Another process needs to be added after Process1

Discovering what the problem is will entail experimentation.

To Summarise

By answering these five questions you will be able to get the most out of your campaign statistics. You will be able to draw meaningful conclusions and refine your communication to deliver the desired results. You will also be able to streamline your control process and generate better analysis and thus fewer decision errors and faster reaction speeds.

Remember:

Make note of communication changes and add them to your statistic

Make note of database changes and realise that you need to compare apples with apples

Make note of environmental changes that will have a serious impact on you metrics

Keep statistics the larger your sample and numbers of observations the more accurate your analysis

Have a great festive season and we'll chat in the New Year.

Darren Reardon
Co-founder
Low Fat Digital Communications
Web: www.lowfat.co.za
Tel: +27 (0) 463 8517
Email: darrenr@lowfat.co.za


Taken from the Low Fat Digital Communications newsletter by Darren Reardon, co-founder of Low Fat.