#MeetTeamBacke: Koree Ritter

A/B testing in email marketing: What
happens when you lose your “control”?

Email marketing lets you do a lot of testing – trying different strategies, lists, and creative to see which combination works best. If a particular formula outperforms others, it’s considered the “control” – until an even more effective approach knocks it off its pedestal.

But, what if there’s no clear control? What if your audience isn’t big enough to produce statistically significant results? And is it possible that, with so much testing, some things can get lost along the way? I asked Koree Ritter, Backe Account Supervisor, for her insights.

Moss: What is your role here at Backe, and how does email marketing figure into it?

Koree: I help guide our account management team, making sure strategy is implemented correctly, clients are well informed and taken care of, projects move forward as expected, and everyone has the right communication internally. The majority of our clients rely heavily on email outreach to stay in touch with current customers, and also use paid email lists to target prospective customers.

Moss: How do you generally approach A/B testing?

Koree: We always recommend at least testing subject lines. After all, you want your customers to open your emails, and some subject lines accomplish that much more successfully than others. But if we’re renting a big enough list, we might test a lot more elements to assess the validity of the creative and see what sparks the interest of the audience. It might be something different than we anticipated.

Moss: Supposing your audience isn’t big enough to generate statistically significant results?

Koree: It can be challenging. But if there’s even a slight difference in the response, even if it’s not statistically significant, we still try to interpret what factors might have driven that difference, and see whether it could drive a strategic recommendation that we can make to the client.

We ask ourselves, “What are the data telling us? How can we analyze that? Based on our own expertise, how do we think our results have anything to do with creative, or lists, or audience?” Then we can go back to the client and say, “This is what we’ve learned, this is how we’re interpreting it, this is how we think it might affect everything moving forward.” It helps our clients trust the direction we’re moving in – even if the data’s not giving us this wonderful big “aha” moment.

Moss: What if you conduct a test and there’s no clear winner?

Koree: If we get good results with both of our test groups, then we try to explore what did well, and tease that out a little further. Whereas if nobody’s opening our emails, then either there’s been a failure in our lists or we need to revisit our strategy and make sure what we’re trying to communicate actually makes sense.

Moss: Do you ever feel marketers can be too preoccupied with testing?

Koree: Yes, I think you have to find a balance. Collecting data is good, and it’s informative; but if you change too many variables, you can sometimes lose sight of the bigger picture: “Who is our audience? What do they believe, and what do we want them to believe? How can we communicate what’s special about our brand?” You don’t want to lose sight of the fact that your audience are people with emotions, not just numbers on a page.

Moss: What’s your number one piece of advice for email marketers?

Koree: Know your audience and know your brand…and you may glean some of that knowledge from email testing and user behavior. Pay attention to your audience, listen to the data, and use it to inform your strategy and your outreach plans.

You might think your audience wants an email a day, but maybe they really don’t. And in fact, you may be desensitizing them to your brand, or creating an impression that may not be favorable.

You can’t force people to engage with you via email. You just have to listen, and meet them where they are. You really have to walk that fine balance of being there when they need you, but not forcing yourself on them.

Stay tuned to Backe social, when we unravel the mysteries behind an ad team’s every day work – from design to strategy to social media and more – in our #MeetTeamBacke series.

Comment on this article using #TalkBacke
Find us @BackeMarketing