What is Email Testing — And why it matters [Deepdive]

Email marketing is an excellent medium for brand awareness, and customer retargeting. It has the highest ROI for your time spent on content since you're not at the mercy of an algorithm to get visibility. But since it's so great, you need to test your emails. Seriously.

3 billion people still read emails everyday!

Deep-Dive into Email Testing

There are several aspects when it comes to testing your emails, and we'll go through the current hurdles and how to possibly fix them. Note that since email has a great ROI, it also has the highest competition.

An average business person receives 10–40 emails per day, but spends only 10–15 seconds to read an email.

Here are the top aspects of email testing you need to consider before hitting send.

Email Deliverability Testing

What's the use of designing a great email if it doesn't reach the person you want to reach? Modern email clients like Apple and Gmail go to extreme lengths to make sure users don't get spammy emails.

Gone are the days when you could just blast emails to everyone in your un-qualified no-opt-in lead list and could expect a 70+% open rate. These days you're lucky if you reach a 45% email open rate.

Fixing email deliverability issues using the following methods:

• Do not share your email server's IP address with spammers - using email marketing platforms with dedicated IP addresses has been correlated to high delivery rates. To check whether your email domain is blacklisted, use mxtoolbox.com & mail-tester.com.
• Not using excessive spam keywords - here's an updated list of top spam words you would want to avoid in 2022.
• Reduce emails going to spam by asking users to double-opt-in entry to the email list - When users submit their email to you, ask them to confirm their entry through a 'confirmation opt-in email'. Not only does this reduces the chances of your future emails going to spam, but it also increases the psychological interest and commitment of the user towards your brand.

Please note while these tools can help improve your deliverability rates it's going to constantly need good qualified lists, and email list cleaning to keep it high. Big corporations go as far as to hire people just for ensuring deliverability stays high.

A/B Email Testing

A/B testing in email has been a great way to run experiments on what works great for your brand. Simply put A/B email testing is where you show 50% of users different content and other content to the remaining audience. Using this technique, you can measure what works best for your email list.

Here is a list of things you can A/B test in your next email:

Switching the Email Sender Name:

Currently, the common trend with a lot of email lists is to include the first name followed by the brand. For example, 'Sara from Crewcharge'.

However, you can boost open rates by commonly switching out the names of different teammates. Remember, it's easy for people to fall in love with people and very difficult for them to fall in love with a brand.

Experimenting with different Subject lines:

Here are some subject line pointers you must always keep in your mind:

• Including the receiver's name has proven to be effective in improving open rates.
• Keeping it short, sweet yet intriguing is key for a great subject line.
• Emojis 😍 are usually perceived to have a good effect on open rates.

The problem with a one-fits-all solution whenever you're dealing with marketing or sales is that humans are great at pattern recognition.

It means something that works once for an email list because of the novelty fails to get consistently high open rates the next time because it's no longer new for your audience. So throw out just quickly copying subject lines or using templates.

A/B test the Preview text:

The preview text is what's shown after the subject in many email clients and notification bars without opening the email.

Over 75% of the people when interviewing with InboxPirates admitted to the fact that they read most of their emails in 2022 from the notification bar which included the sender name, subject and preview text - with no HTML.

Most email marketing clients allow setting a preview text to the emails, if not you can insert a div with zero visibility and font-size 0 to include it on any email sending provider of choice.

Playing Switcharoo with Email CTAs

If you're a SaaS company looking to activate leads or an agency looking to scout new clients - your email call-to-action (CTA) is generally the biggest factor that decides the result of your email.

Building a great call-to-action comes in three simple steps:

• Make sure clicking the button is barrier-free and value-rich for the receiver.
• CTAs should be self-descriptive of what's about to happen next.
• Give a sense of exclusivity (Only for you) and/or scarcity (Only for the next 5 days) following the AIDA principle.

Readability & Accessibility Testing

While sending emails it's extremely important to note the readability & accessibility of your emails to ensure maximum reach. Over 70% of the web is inaccessible to people with special needs.

At Inboxpirates we help you test emails for:

• Read time of your emails - Generally speaking, you must aim for a read time of 1–2 minutes.
• Word count & Sentence count - Aim for a word count of 100–200 and a maximum of 20 sentences. This has proven to have a maximum impact according to Hubspot.
• Word choice, Spelling, and Grammatical Errors - Apps like Wordtune and Grammarly help you avoid grammatical errors and spelling mistakes.
• Hemingway readability score - Aim for choices of the words in your email that a sixth-grader can understand.

The ego-centric bias & the happy path fallacy

While most email service providers like Apollo, Sendinblue, Hubspot, or Mailchimp give you an inbuilt email preview tool, they mostly only show you the aspect of the mail being opened in your receiver's inbox on the full screen of their device.

While this view is great for just validating the HTML on smaller devices, it rarely captures the customer journey to your email or how it's read most of the time (which is from the inbox view, from a preview pane, or even from the notification bar).

I call this the ego-centric bias because it makes you feel as if it's the only email in your receiver's inbox and let's be honest - that never happens.

Combine this with the fact that these tools support your assumption that you push an email out at 8:00 AM, and your readers read it as their priority by clicking the only notification directly opening your email in fullscreen.

In reality, we know for a fact that people get a minimum of 10 emails per day, meaning the notification bar is first crowded with other emails combined with notifications of other apps, and then the actual open might occur later from the inbox view filled with lucrative email previews from your competitors.

Inbox Preview Email Testing

Inbox preview email testing refers to seeing your email in the inbox view or the email list view where your email is stacked up against several other emails to see where your email content, the email preview text, and the subject line get cut off.

This is a great way to eliminate the ego-centric bias and acts as a realistic email testing metric to measure open rates using A/B testing.

The Problem of Email Clients Taking Control

Email clients like Gmail and Apple mail have a huge task on their hands - it's to make sure their readers have the best experience while reading your email. They feel your email must be designed in such a way that your design fits into the overall aesthetics of their application.

Here are the common ways your email design is overridden and why it's important to test your emails:

• Font sizes are usually not respected by clients like Gmail on Android where everything is scaled by up to 50% so it's readable to the user.
• Whenever your images are above the width of the device, they are scaled down to fit the screen and if the aspect ratio is too high, it might be completely invisible.
• HTML tables that are used in emails for responsiveness are always scaled down to the width of the device and it's hard to predict how they might look in different screen sizes.

The forced-dark-mode situation with email

Last and certainly the biggest offender is the ambiguity caused by clients like Gmail, Yahoo, and outlook that completely invert the color of your email using a propriety color table.

There's no control over designing dark mode-specific emails. Most email clients do not support media queries for dark mode. While some clients have hinted at supporting media queries in the future, the most popular email platforms like Gmail or Yahoo which constitute well over 39% do not support CSS media queries for dark mode yet.