Testing is the Foundation of Database Marketing

Every marketing action is done with the expectation of some beneficial outcome. The ultimate measure of success for an organization is sufficient sales and profit to succeed in the moment, keep pace with the market, and be around in the future. However, there are a lot of intermediate goals to which marketing actions contribute.

Resources are scarce. It is the responsibility of a marketing manager to perpetually seek the optimal marketing mix. What is the optimal marketing mix? It is the combination of marketing elements that results in achieving the company’s/organization’s goals at the lowest cost.

Every marketing action is a choice based on explicit or assumed results expectations. A problem for marketing managers is that measuring the contribution to success of any element of the marketing mix is very hard … except in the case of direct marketing. Every direct marketing campaign that goes to a targeted population embodies the opportunity to measure the direct response. Each piece sends people some information and then asks them to do something, usually in the very near term.

Every marketing action is a test. Do people respond as hoped/expected? Every possible testing layer can reveal valuable information to a marketing manager. Here are a few of the layers to consider:

1. Although it is seldom checked by marketing managers, testing what they are doing against doing nothing is the basic test of every marketing action. Sales that are the result of marketing efforts rather than doing nothing constitute what is called LIFT.
2. Early in my marketing education, it was pointed out that the quickest way to kill off a bad product is to advertise it. Lots of people try it, won’t like it, and sales crash. The basic presumption, however, is that any marketing pressure in support of a product or service will produce beneficial results. Response analysis of a fixed piece, often referred to as the CONTROL, over time establishes what can be expected from that piece. If the control is left unchanged for long periods of time, a number of analytical interpretations are possible.
   • The most basic question response analysis answers is, “Did the campaign achieve the return on investment/profit goal for marketing investments, at least pay for itself, or lose money?” One measure that can easily be compared across marketing investments is the advertising to sales ratio. “A/S” divides the cost of the campaign by the sales attributed to the campaign. Many firms manage their allocation of marketing funds by comparing this ratio. In conjunction with the magnitude of the results, measured by the ratio of attributed sales divided by total sales in the response period, this method works well.
   • Variations in response can be checked against general sales patterns to determine whether consumer responsiveness to direct mail is seasonal or sensitive to other short- or long-term factors such as weather/natural disruptions, political cycles, economic conditions or world events.
   • Variations in response by market segment can be checked. In this way, it can be determined not only if the piece is working well overall, but with specific groups of potential customers. If the control is found to not work with some segments, mailing to the non-responsive segments might be stopped to save resources or this could be an indication that something needs to be changed to appeal to these segments.
3. At this point a key distinction must be made about testing. Any pile of data can be analyzed to get enough directional insight to decide if something really needs to be checked out. Unless the findings from the directional analysis are so overwhelmingly skewed to one interpretation, directional findings should be used to guide the development of a true test of a hypothesis. True tests compare two or more elements against each other in a subsequent campaign designed to be both statistically significant and uncompromised by variations not being tested. It should also be pointed out that direct marketing pieces present a means to test marketing messages and offers on specific market segments that will be used in other media.
4. Every direct marketing campaign represents an opportunity to use testing to seek improvements to the control or discover that some market segments merit their own campaign distinct from the general one. Various printing techniques make it possible to test a wide range of elements in every aspect of direct marketing pieces. In order for a test to be valid, a number of conditions must be met. Key among these are:
   • No inherent bias. The group of people receiving the control and the test must have the same composition or else adjustments in the analysis process are needed to balance the results. It would not do to test a new design element by sending it to a select population and send the control to people not sharing the same characteristics as the select population … rich people, for instance.
   • The sample size for both the control and the test pools have to be big enough to achieve statistically significant results. There are different ways of making this determination, depending upon the statistical method being used to assess the results. The simplest statistical method for comparing the response rate of a test pool to a control pool is called the Binomial Test. There are formulas that calculate the required sample size needed for valid test populations based on the historic response rate of the control.
5. When is a good time to initiate testing? Because every database marketing campaign is effectively a test, it is better to think of direct marketing as perpetually testing. Take advantage of every flight to test or analyze response in order to gain knowledge that can be used to guide future tests. Because of company budgeting and sales objective cycles, it is good to develop a testing schedule that will allow marketing managers to take advantage of the best knowledge available for what are likely their peak sales periods. (Nearly every business has some seasonality.)
   • For many small businesses, testing seldom gets far beyond the directional level, but it is still important to be timely. Small changes in a control can result in big benefits. It happens often enough that managers have a misconception of who their customers are and what they think that customer profiling, market segmentation, response analysis, and testing are needed to get managers back in touch with their customers.
   • For large businesses, the opportunity to be perpetually testing at a large scale for a variety of attributes exists. Statistical methods beyond the Binomial Test make it possible to test many campaign elements across a wide range of market segments, if the mailing counts are large enough.
   • In order for testing to become an institutional practice for a business, management has to appreciate that learning new things about its customers and how they think is more valuable than the slight loss the business might take on some test pools relative to the performance that might have been achieved sticking to the control.
   • In a sense, the time to engage in testing or analysis to support testing is always NOW.
6. What can be tested? Just about everything:
   • Graphics: Colors, shapes, images, font types and sizes, etc.
   • Formats: Letters, cards, with or without coupons, stamps, windows, envelope sizes
   • Messages: What are the important issues going on in the minds of consumers relative to the product or service that need to be mentioned so that the product will be recognized as fulfilling a need?
   • Price: Price points matter, especially to households at different income/wealth levels.
   • Offer: The human mind is a puzzling thing. Sometimes having a lower threshold to qualify for an offer will result in more customers responding and customers spending more money than a higher qualifying threshold. Testing looks for the offer that gets the best overall results.
   • Target market segments: Even for the same product, different market segments may think and respond differently to one piece rather than another. Better results come from touching the two segments with different marketing communications, not the same piece for both.
   • Timing: Is there seasonality? Is there a monthly pattern? Is there a weekly pattern? When in these cycles is the best time for a direct marketing piece to reach a target audience so that it fits into their path to purchase, information gathering, evaluation, and decision-making behavior patterns?
7. Companies and their products build equity not only in physical assets, but also in good will and brand identity. The term that best captures brand identity is marketing position. Every marketing communication should in some way reinforce or expand the positive dimensions of a brand’s marketing position, without over-extending its claims and creating confusion. Again, the human mind can only keep track of so many ideas and associations. It is best to keep a brand’s marketing position simple.
   • Every marketing communication trains those who see it about what to expect from the brand and its products or services.
   • Testing helps identify what is most important about a brand’s marketing position to customers. Testing also helps track how successfully each marketing communication connects with potential customers and causes them to make choices the company is trying to train people to make.
   • Testing and analysis of sales and customer behavior is what keeps marketing managers from getting distracted by shiny objects. The brand and its marketing position are not well served by constant fluctuations in style, message and value proposition, unless being unpredictably new and different all the time is the brand’s marketing position. (Generally, not a winning strategy for consumer products.)
     • Testing supports incremental controlled change and constant improvement.
     • The measurements supporting testing also help marketing managers recognize when market demands are changing so that the brand can ease along with the market and not experience catastrophic disconnects.
8. Now, when many consumers are experiencing a disruption in their lives is a time when reshuffling the elements of a brand’s marketing position to emphasize what is newly important to people is important. Making these changes should be based on testing and consumer research, not guessing.

Contributing Author:
Clinton Kennedy
Senior Database Marketing Analyst