Does E-mail Open Rate Still Matter?

For many years, one of the most frequently monitored metrics of the effectiveness of email campaigns has been the open rate, i.e. the ratio of messages opened to messages sent. With an update to Apple’s Mail Privacy Policy (MPP), the debate has once again heated up about the reliability of the results reported in this way.

The lack of reliable open tracking for Apple Mail users has made some marketers realize that they need to rethink the way they analyze the results they receive, assess the performance of their campaigns and draw conclusions about deliverability. Currently, many are still basing their reports to their superiors solely on one metric – opens – so their drop compared to previous months involves an intensive search for the reason for this.

Is it worth continuing to look at open rate as the main email campaign performance metric?

Very often we are asked by our clients to help analyze the results they receive, find the reason for the sudden drop in open rate in recent email campaigns and support them in keeping it high.

Why has the email open rate never really been important?

In an ideal world, every opening should be reported when the recipient of a message clicks on it himself. However, mailbox providers do not provide direct information about the performance of such an action. To be able to monitor and count each such event, it is necessary to use a reporting system that enables this.

Most often, Open tracking monitoring is based on one (or more) of the following methods:

• tracking pixels *

• downloading styles or fonts

• clicks on links (thanks to Link Tracking)

• requests for confirmation of receipt

• replies to correspondence

* In the HTML code of an email, usually at the very end, an invisible, transparent image (usually 1×1 pixel in size) is embedded. When the message is opened, all graphics are loaded – the tracking pixel is one of them. In addition to the time it is downloaded (counted as the time the email is opened), other information such as geolocation, IP address, device type or operating system used may also be logged.

It might seem, therefore, that by using Open tracking, we will obtain accurate and real information about the opening of each message. Unfortunately, this is not really accurate – the data received very often does not reflect the actual activity of the recipient. The reasons for receiving unreliable, “falsified” results can be:

Automatically loading or blocking the display of images

Some mailboxes automatically load images, while others on the other hand require an action from the user  (e.g. Outlook blocking the display of graphics by default for security reasons.) The tracking pixel will not be downloaded until you allow it by clicking on the appropriate options dialog box.

Another example is the aforementioned Mail Privacy Policy. As a result, the email will first go to Apple’s servers, automatically downloading all images, and returning the incorrect information that it is end-user activity.

In addition to preventing the tracking of opens, Apple is making it possible to hide IP addresses from which the connection is made (no location), email addresses and the device type used.

Spam filters

Scanning messages for threats such as false links or dangerous content (malware) can automatically load images (including tracking pixels) or log clicks on links

Bots activity

Some mailbox providers (like Gmail) trigger opening via bots. Most often, these are multiple opens, registered within seconds of receiving an email

Autoresponders

If your system also relies on responses from recipients to count opens, receiving an autoresponder can disrupt the accuracy of reporting – especially during periods of increased absence – such as holidays or Christmas break

Browser plug-ins or tracking pixel-blocking software

The characteristic attributes of a tracking pixel – such as its 1×1 dimensions – make it easy to locate it in email and block it by dedicated software.

Excessive length of the message

If the email is too long, it may happen that a pixel will not be downloaded at all because the message will not be displayed entirety. The tracking pixel is usually located at the very end of the message and Gmail for example clips the message when it exceeds 102KB

Which metrics are worth analyzing?

Now that it has been discussed how easy it is to get a falsified result for the Open rate, the question arises – is what has been measured so far completely useless?

Not really.

When evaluating the success of performed mailings, it is important to look at the results multi-dimensionally. In addition to open rates, we should also analyze other metrics that will allow us to monitor in a more sophisticated and aware way how the campaign was perceived by the recipient and whether it had a positive impact on the conversions. Things to look out for include:

• Website traffic – the number of potential customers visiting the online store website measured over a certain period after the campaign is sent out

• Number of orders placed – from redirection via links in the mailing

• Contact database size (growth of newsletter sign-ups and unsubscribe rate) – a high UTOR (Unsubscribe To Open Rate), i.e. the number of people who unsubscribe from the database after opening a message, may indicate, among other things, errors in the design of an email, offer or database hygiene

• Spam Complaints Index (Feedback Loops), which assesses how many people marked an email as unwanted by clicking on the “Mark as spam” button

• ROI – which is a profitability indicator expressing the return on investment in percentage (profits concerning the funds invested)

So how to use the Open rate?

First and foremost, it is important to remember that open rates, as well as other email metrics, should be assessed for their volatility – so take a closer look at any significant fluctuation.

If a recent email campaign has, for example, a 15% lower Open Rate than the average performance of the last month, this is a clear signal that this particular campaign should be checked – email design, subject line, preheader, offer (and personalization), but also the prepared audience segment. 

Most of the potential errors can be avoided by performing A/B tests beforehand. 

The same applies to Open Rate per provider results. If there is a drop in the results for one particular mailbox provider, it may be a sign of some deliverability problems, e.g. throttling (limitation of emails intake by ISP) which occurs in case of a sudden increase in volume, spam bounce as a result of a violation of the provider’s internal policy or even a change to a new IP.

It is worth bearing in mind that the Open rate should be analyzed not only as an indicator measured over time but also in comparison to other metrics. One less frequently opened email campaign should not be a cause for concern. However, if we observe a sudden, significant drop in open rate with a high number of hard bounces, this is a clear signal that something is wrong.

Summary

Opens are not a useless indicator when it comes to measuring the effectiveness of email campaigns, but they should definitely not be the only metric we analyze. The number and reasons for message bounces and spam markings are far more valuable sources of information on how your messages are perceived.

Use the open rate to verify other elements, such as the classification of messages into appropriate tabs (by using Seed List). The open rate should not be the most important metric when assessing deliverability, there are better ones for this. When evaluating the success of a campaign, therefore, look not at the number of messages opened, but at the real profits that engaged recipients are generating.