How to Tell If Your Promotions Are Driving Visits That Would Have Occurred Anyway

The aim of many marketing efforts is to drive incremental guest behavior, like increasing guest spend or driving additional visits. But, how do you know if the behavior is truly incremental? How much of the lift is due to the campaign versus behavior that would have occurred anyway? Are we giving discounts to guests who would have been willing to pay full price?  These are questions you may have heard from your finance team, franchisees and operators.

Reporting on incremental behavior can be difficult to answer with general marketing media such as TV and radio spots, billboards, or Internet banners. However, because of the one-to-one nature of your loyalty program, these are questions that can and should be answered.

To address these questions, we recommend implementing “target and control campaigns.”  Target and control campaigns allow marketers to conduct real world experiments where the results are clear and easily understood.

Testing offers requires two groups that exhibit identical characteristics: A target group and a control group. First, define the population you want to target in your campaign. Your target audience could be just about any segment within your data such as all members who haven’t visited in the last 3 months, or all members who joined your program during the last quarter for example. Once you have defined your target group, the Paytronix software will automatically segment a randomly selected subset of the target population. ­­During the length of the campaign, the control group should have the exact same experience as the target group except for the offer you are testing. In more advanced tests, you could further segment the test cell groups into sub-segments based on offers.

During the promotional period members of both the target and control group will engage with your brand. We can compare the two groups using any number of metrics, such as response rate, check size, visit frequency, total spend, and more. When the campaign period is complete, you can continue to track the performance of the two groups to see if the campaign had any lingering effects.

In order to better understand how setting up a campaign using a test and control group framework helps us, let’s take a look at the following campaign:


Sample Test Campaign

Assume there is a group of guests who come in once per month on average. In September they all came in one time as usual, but in October 5,000 of them did not! For these 5,000 we want to test an incentive of $5 off of their next visit. We will use the target and control methodology to measure the effectiveness of this campaign.

Assume we observe the following outcome:

The table above shows what we observe when splitting the 5,000 guests into a 4,500 member target group and a 500 member control group. As you can see 33% (1,700) of the target group visited during the month, while only 24% (120) of the control group visited during the month.

Furthermore, the target group generated 2,210 checks (1.3 per visitor) during November while the control group generated 144 (1.2 checks per visitor). The target group spent $35,000 in November, while the control group spent $2,880. So on a per check basis the average check size was $15.83 for the target and $20.00 for the control.

In order to measure the incremental spend for this campaign we should compare spend per targeted figures for both groups. For the target group, spend per targeted guest was $7.78 and the control was $5.76. So the incremental spend was $7.78 – $5.76 = $2.02 for each of the 4,500 people targeted. Then we should multiply the $2.02 * 4,500 (test group population) in doing so we get $9,090 in incremental revenue that was generated by this campaign!

The target and control group, enabled us to readily answer the question “how many people visited without the offer?” Without the target and control campaign in place, we would have overstated incremental revenue since we would not have had insight into those visits that would have occurred without an incentive.

In summary, using target and control groups provides a better way to measure outcomes of your real-world experiments. They provide consistent, accurate, and relevant data to help marketers reach their goals.

Chris Randall

The Author
Chris Randall is an analyst on the data insights team. He has a passion for: helping firms understand what incentives are useful in order to drive consumer behavior, as well as developing models used to identify and predict trends in guest activity. He holds graduate degrees in applied economics from the University of Cincinnati as well as the Norwegian School of Economics (NHH).

Did you find this post useful?

Leave a Comment


Reload Image

No comments on “How to Tell If Your Promotions Are Driving Visits That Would Have Occurred Anyway”