Cold email has been a hot topic for both digital marketing firms looking for more leads and affiliate marketers looking for link building opportunities. But, for those with more experience in SEO and SEM, email marketing is a whole new beast to explore. It's such a big segment of digital marketing that there are entire courses devoted to it.
So, I figured the best way to do some split testing with cold email is to break up the whole process into very small sections. For this test, the main concept is to test a few different subject lines and see which one has the best open rate.
This is just a simple case study. It's only testing the subject lines of initial cold emails.
I also only targeted one specific niche: family therapy. Other niches may yield slightly different results.
There is also a fairly small sample size, but it should be enough to get some useful takeaways.
This cold email outreach followed a very simple design. Here are the key elements:
As you can see, it was a very vanilla and straight froward design. As far as the subject lines tested, they were chosen based on offering the recipient a slightly different idea of what the email might be about.
Here are the three subjects that were used and why.
Subject Line 1 - Quick Question
I wanted a very generic subject line. It was short and did not really give the recipient an idea of what the email was about. It did however suggest I was asking them for something instead of me trying to jam some unsolicited advertising in front of them.
Subject Line 2 - Help More People
The second subject line was more revealing in that it targeted the recipients' businesses. Since I contacted family therapists, helping people is something that is of interest to them.
Subject Line 3 - Help More People?
The last subject line tested was a combination of the first two. It was targeted and not generic, but it included a question mark to make it a little softer in tone than Subject Line 2.
Before looking into the results, we can make some assumptions about what to expect.
My assumptions were:
These are both pretty safe and simple assumptions considering the differences in the subject lines.
Enough words, let's look at the stats.
The only fields that are important for this test are "Sent", "Opened", and "Bounced".
As we can see, of the 100 sent emails, 4 never made it to a live inbox. We still managed to get 96 emails through, and they had a combined open rate of 56%.
That's not too shabby. But, how does the generic subject line match up to the other two?
Help More People
There were a few more bounces here than with the first 100 emails sent. Those extra bounces brings the total delivered emails down from 100 to 93.
That's still a decent amount of emails hitting inboxes though. But, the open rate here was considerable lower at 48%.
That's almost a double digit drop off from the generic subject line.
There's still one more to check out.
Help More People?
These results look extremely similar to the generic subject line. They both had 4 bounces, and the total opened percentage is very close.
For this we got a 55% open rate. That's not bad at all for a cold email and basically only one open less than the generic subject.
These was a very small but targeted case study, so we don't have tons to look at here. But, we did get some interesting and in my opinion unexpected results.
Let's see if my initial assumptions were accurate -
Assumption 1: The generic subject line would have the worst open rate of the three tested
Wow. I could not have been more wrong. Not only was it not the worst of the three, but it was the winner.
I have a feeling by keeping the contents of the email vague, but by making the subject line more about asking for something instead of pushing something, it tickled the curiosity of the recipients.
Assumption 2: There would be little difference between "Help More People" and "Help More People?"
Who would have known that adding something as simple as a question mark would make such a big difference on open rates? But, from this test we had almost a double digit difference - 55% vs. 48%.
I feel even with the small sample size, that is a pretty significant difference.
Overall, this is a fairly small scale case study. However, that doesn't mean we can't take away some useful information.
First of all, it's clear that subject lines can have an effect on open rates. Most people know this already, or at least believe this to be true. Having some stats to back it up though is always nice.
The bigger takeaway though is that the exact wording of your subject line might be less important than the general tone.
It's pretty clear that "Quick Question" and "Help More People?" are very different in their wording. But, they share a tone that piques curiosity unlike "Help More People", which is just basically a statement.
That being said, I'm still shocked at just how wrong as I was in my assumptions. I guess the biggest takeaway of all is that no matter what your assumptions are in any aspect of your marketing work, unless you test them you'll never know if you are correct or not.