Unleash the Power of Market Testing in Your Membership Recruitment

Claude C. Hopkins is acknowledged as the great grandfather of direct marketing. In 1923, Hopkins wrote Scientific Advertising in which he declared that: “The time has come when advertising has in some hands reached the status of a science.” 1

His fundamental marketing thesis was: “We learn the principles and prove them by repeated tests. This is done through keyed advertising by traced returns . . . We compare one way with many others, backward and forward, and record the results. When one method invariably proves best, that method becomes a fixed principle.” 2

Today, his premise of testing is as true as ever. In fact, testing may even be more important now because the vast array of options available to marketers. A test can mean the difference between a stunning success for a product or an abject failure.

And because of the larger quantities and costs involved, testing is a particular necessity and an ideal opportunity when it comes to membership recruitment.

It is not uncommon to see a successful test change recruitment response rates by the following percentages:

• List tests – Can impact response by 500 percent.
• Offer tests – Can impact response by 200 percent.
• Creative tests – Can impact response by 100 percent.

These test outcomes highlight that by doing the same old thing over and over again, there is likelihood that a marketing program is substantially sub-optimizing the potential returns that could be achieved.

If testing is so important, then how should it be done?

There are two aspects of testing. Let’s call them the “art” and the “science” of testing.

The art of testing involves thinking outside the box and creating a new way to do things. In a market driven organization, each project should start up with a brainstorming session that asks: “What if?” or “How about?”

Bob Stone, in his landmark book, Successful Direct Marketing Methods, suggests creative helps like the following questions to get the thought process going. He recommends asking:

• Can we combine?
• Can we add?
• Can we eliminate?
• Can we make an association?
• Can we simplify?
• Can we substitute?
• Can we reverse? 3

Once a good set of test options has been developed, it is time to prioritize them. The key here is to test big things. Look for a breakthrough in testing. Too many testing dollars are spent on inconsequential testing – like who signs the letter. The fact is that testing small things will have such a small impact on the results that chances are good it will not have statistical validity.

In addition to brainstorming, there are some specific high leverage areas to consider testing. These high opportunity areas include the following:

• Lists – One of the easiest and most productive tests is trying new lists. For a full discussion on testing lists, take a look at my post, Five Strategies for Picking the Best Marketing Lists.
• Frequency – Try marketing more frequently to top prospects and customers.
• Pricing and offer – Psychological price points are for real. As a rule of thumb, a price ending in a “7” or “9” will generate more orders and dollars. You can test price points by offering a special acquisition dues discount to new members.
• Packaging – Test a bundled membership product instead of selling a one size fits all membership product.
• Media – Many media are available today and need to be tested. Try direct mail followed by an email linked to a microsite compared to a stand alone email or mailing.
• Messaging – Try new messages that emphasize a different value of membership. An easy and fast test of messages can be done with email subject lines. Send out a small portion of the list with a variety of subject lines. The group with the highest open rate wins and that subject line is used for the remainder of the list.

Equally important to the “art’ of testing is the “science” of testing. The science of testing starts with creating proper test structures. The key here is establishing a control package and testing against it. This is done by drawing a portion of names out of the control group of the marketing effort and using them for the test. Then structure the test by holding everything else constant except the variable that is to be tested. For example, if the test is for a special discount offer, then on the test segment use the same format as the control package and mail the test promotion to an equal ratio of the control lists in the mailing.

A test obviously does not always produce better returns than the existing control – that’s why to lower risks tests only go to small segments of a larger promotion. However, statisticians tell us that in each test cell we need a minimum of 40 paid responses to give us a statistically valid test. Therefore the number of anticipated responses will dictate the size of each test segment. If a 0.50 percent response rate is expected then the test cell should include a minimum of 8,000 names (40 / 0.005 = 8,000).

The other challenging yet critical component of the science of testing is tracking. Despite the difficulty in tracking, the bottom line is that the potential returns of testing are so great that one way or another an organization needs to build some kind of mechanism to track returns. Each organization will have to work with its computer staff and order processing staff to find the best way to track returns from a test. However, there are some methods of tracking used by other organizations that have performed well. These include:

• Laser Personalizing, ink jetting, or labeling the reply form with a keycode and requiring the form to be returned to receive a special offer.
• Performing a computer match between the returns from a given period of time and the keycoded mailing lists that were used in a mailing.
• Programming in a “special order code” in order forms on web sites that are required for special pricing or offers.

Once returns come back in, compare the responses in the control group against the test cell. Look to see which cell generated a higher return on investment and make the best performing test your new control.

Testing is an ongoing process. Over time, it becomes part of the culture of an organization. A focus on testing ensures the flow of new ideas and new members that an organization needs to keep growing. And it also provides a methodology for validating each of these new ideas.

To quote Claude Hopkins again, in the past: “Advertising was then a gamble – a speculation of the rashest sort. One man’s guess on the proper course was as likely to be as good as another’s . . . That condition has been corrected . . . Advertising has flourished under these new conditions. The results have increased many fold just because the gamble has become a science.” 4

Testing in membership recruitment allows your organization to market more efficiently. And if Claude Hopkins could do tests back in 1923, I am sure that there is a way to build testing into your membership marketing efforts.


1 Claude C. Hopkins, Scientific Advertising, NTC Business Books, 1991, page 213
2 Ibid, page 215.
3 Bob Stone's landmark book, Successful Direct Marketing Methods, 1997, NTC Business Books
4 Claude C. Hopkins, Scientific Advertising, NTC Business Books, 1991, page 217-218