Let there be churn

It’s a safe bet that if you ask a deliverability expert about the holiday season, they would almost all tell you to expect more list churn than in non-holiday time.

As many senders open up their flood gates, recipients are suddenly getting a lot more email marketing than usual, and during a small timeframe. So, naturally, the theory is that there will be more churn than usual. I wanted to test this theory, so for this blogpost I looked into our logfiles and other metrics to determine if that theory is actually true. Let’s find out.

My methodology

Disclaimer that this is not a regulated scientific study, this is simply me looking at numbers available to me.

To be precise, I looked at two time frames. Time Frame 1 (Holiday season) was the most intense four-day period possible, Black Friday to Cyber Monday. Time Frame 2 (Control Group) was a regular Friday to Monday weekend in January.

I took the overall volume of mail sent, as well as the recorded bounces, feedback loops, and unsubscribe requests.

For further clarity:

Volume: The total volume of all mails sent determined by our mail server logfiles, either being successfully delivered or having bounced.

Bounces: All mail being rejected by the receiving mail servers, either temporarily or permanently.

Feedback Loops: A Feedback Loop (FBL) is a mechanism where a Mailbox Provider may decide to sent a notification to the sender when the recipient hits the “this is spam” button. Not every receiver does send an FBL, out of the big providers it’s Microsoft and Yahoo!. Dotdigital automatically processes these FBLs and suppresses them for the account the mail originated from.

Unsubscribe requests: all successful requests our very own unsubscribe pages registered, including the standard unsubscribe page as well as the ones created by “Pages and Forms”. Also included are requests via the “list-unsubscribe” header if the mailbox provider in question is using the HTML version of it.

Is there churn?

Drumroll please… the answer is yes. And no.

Let’s start at the beginning, with the unsubscribes. For the two timeframes we do see quite a big difference in unsubscribes, 0.1% for the Black Friday weekend and 0.06% for the control group weekend. This result is very much in line with my expectations.

Next we look at the complaints by Feedback Loops, and while the percentages are slightly in favor of the busy period, it’s really a very small difference of 0.022% vs 0.019%. This is very much a testament to our customers for building a strong reputation and relationship of trust with their recipients, so that their subscribers are more likely to trust them to honor the unsubscribe instead of just hitting the spam button.

And finally we look at the bounces, and again there is only a very small difference. This time the numbers are slightly in favor of the Control Group by 0.05%, which tells us two things. Firstly we are able to see that Mailbox Providers were well prepared for the increase in mail during the busy period, and also that our clients were not suddenly choosing to mail to all available recipients, and instead followed best practice.

Summary

My gut feeling that there would be higher churn of recipients during the busy holiday season was correct. That said, I expected it to be higher than I found. Additionally, after examining the data – seeing bounces and complaints stay relatively the same is intriguing.

The results not being as dramatic as I anticipated could be due to consumer attitudes around this time, where they’re both used to receiving more from brands and are themselves more open, and even excited to receive more emails than usual as they hunt for discounts.

This could also be due to my data set. Perhaps Dotdigital customers are simply reading all of our blog posts and guides on best practices, and as such, are seeing great results. Our work here is done…