How Can I Tell If a Direct Mail Marketing Campaign Has Been Successful?

When creating a direct marketing campaign, it would be great to have a method for searching and identifying competitor’s campaigns that are successful.
Using Who’s Mailing What! is a great place to start to search through, examine, and categorize other people’s efforts. This article will focus on several techniques that will help identify which approaches have worked for others and which ones have had less than stellar results.

Using Controls

In marketing, the term “control” refers to a mail piece by which we can measure the success of future campaign tests. If you have run campaigns in the past, you can identify the one piece which has performed best for you. This becomes your control.

Much like in scientific experiments, it becomes the baseline that we use for measurement. If a future campaign does not perform as well as the control, we make notes on what changes were made, and try new modifications. If it performs better than our control, we replace it with the new control.

Ideally, we can get the best results from our own campaigns, as we know our own campaign history, and we know exactly what we have tried in the past, and we can make modifications based on specific criteria that meet our precise needs. However, we may not always have the advantage of previous campaigns. If we are just beginning, we often need to create a campaign from scratch. So how do we know what is most likely to work, if we have no previous benchmarks?

Controls in WMW!

Within WMW!, we have over 126,000 records of past marketing campaigns from more than 21,000 marketers. While we don’t know the specific outcome of each mailing, we do have information that delivers great value to you.

Organizations (particularly those with large enough marketing budgets) create a set of key performance indicators that they wish to meet with a marketing campaign.

We define controls in WMW! as campaigns that have been mailed repeatedly over time. The simple fact that a campaign was repeated gives us an indicator that it has succeeded at meeting a specific key performance indicator. These particular campaigns are given a “control” flag to help you identify good models of success.

Even more reliable than controls are what we label as “grand controls.” These are campaigns that have run regularly for multiple years. We will typically see that grand controls are run by companies with solid marketing budgets and a commitment to direct mail as a business building channel.

In WMW!, when searching for past campaigns, you filter to include only controls and/or grand controls in the Direct Mail filters:

Below is an example of a control. The campaign was run three times; once in December, then in March, and again in April, suggesting that the previous two campaigns were successful.

WMW! Proprietary Performa Mail Indicator

Another type of control, identified by our machine learning algorithms, is our own proprietary Performa Mail category. We start with all campaigns from multiple organizations within the same industry. Our algorithm searches for offers included in these mailings that are highly similar and where the creative design and format are close to the same.

Most often, these are created by a single agency and tested across a variety of their clients within an industry. We can infer from this type of scenario that a Performa Mail flagged campaign is a winner. It has consistently delivered results, so it behaves like a control.

Let’s take a look at a couple of examples of direct mail pieces that are indicated as Performa Mail campaigns.

Here we have two examples of mailings sent by two separate insurance companies, but that share a number of the same characteristics, First let’s look at the envelope:

Here is offer 1:

And here is offer 2:

We can see that while these mailings are coming from two entirely different companies, they are almost identical. Both have the logo clearly displayed over the mailing address, and both have exactly the same text on the envelope. In this case, our algorithms would give this mailing a relatively high score, as the mailings are virtually the same.

Now let’s look inside the envelopes. here’s the first offer:

Now, let’s look at the other one.

As we can see, these two letters are virtually identical in almost every way. Some text is changed, but mainly it is only the company information. While these two messages are almost the same, they would not have been identified as “controls” because they come from different companies, however, as we can see that two separate business entities are using almost exactly the same messaging (and in this case, down to the exact dollar amounts).

The fact that two different companies are using almost the same campaign suggests that these are effective in producing results. To get a client to sign up, the agency that created these campaigns has proven to each client that this campaign works. Which is tremendously valuable information to us and to you. Our internal algorithms identify these two campaigns as Performa Mail controls; they are like controls, but used by different companies. While we will soon make these available to be searched on the WMW! platform, for now they are included as a component in our Performance Score (see below).

WMW! Performance Score

WMW! has developed a “Performance Score” which ranks mail pieces on the likelihood they will perform well, on a scale of 1-100. All mail pieces are given a score, and you can sort by this score to find the most likely or least likely to perform campaigns.

The scores are unique to each business category and the algorithm considers data such as mail piece format and copy, including offer and headline ranking, weighted by controls, grand controls, and Performa Mail flagged campaigns, as well as the size of the mailing organization, to generate the score. While we do not know the actual outcome of each campaign, using powerful machine-learning tools we are able to infer that one campaign is more likely to be successful than another and assign it a score.

You will be able to see the performance scores in the search results, as well as on the PDF viewer page.

Conclusion

By using controls, Performa Mail controls and the Performance Score in Who’s Mailing What!, you can get a clear picture of what is working for others. By looking at controls, you can see campaigns mailed by organizations that have been run multiple times, which are good indicators of success. With Performa Mail controls, you can recognize which formats and offers are effective across an industry.

By using this information, you will learn what is most likely to produce the best result when setting up your own campaigns.