Question

Topic: Research/Metrics

Test Methodology For Effects Of Lead Nurturing

Posted by Dane Robbins on 10/11/2012 at 1:55 PM ET 500 Points
We're having an internal disagreement about the testing methodology for an e-mail lead nurturing campaign. The goal is to measure the effect of a lead nurturing campaign on overall bookings. The lead nurturing campaign consists of 4 e-mails sent over a 2 week period to those participants who click through on the initial e-mail. The client wants to do a 50/50 split on the database. Both the CONTROL and TEST groups will get the same initial e-mail. From the TEST group, only those who click will continue with the lead nurturing treatment. The CONTROL group will have no opportunity for lead nurturing regardless of CTR.

To compare the groups, the client wants to issue a promo code that is the same for both the CONTROL and TEST groups for the initial e-mail, but different for those who click through in the TEST group and receive the lead nurturing. I would advocate issuing a separate promo code for the CONTROL and TEST group each for the initial e-mail, and then a third promo code for those who book from the lead nurturing campaign. Otherwise, I think combining the promo codes on the initial e-mail would incorrectly skew the control group.

My reasoning is that anybody in the TEST group that doesn't click through is very unlikely to book directly from an e-mail whereas the CONTROL group can click through and book normally. In essence the TEST group would drag down the number of the CONTROL if the same promo code is used for both.

Anybody have any experience or suggestions to handle this properly? Our prior experience and that of many case studies suggests that lead nurturing will increase bookings over the control. But it's important that the methodology isn't called into question.

To continue reading this question and the solution, sign up ... it's free!
SUBSCRIBE
Or, SIGN IN

RESPONSES

  • Posted by telemoxie on 10/11/2012 at 5:47 PM Accepted
    I don't have lots of experience with metrics, but have a lot of experience with lead nurturing.

    I would have a different concern. Trying to read between the lines, I'm assuming that the client wants to know whether a lead nurturing program is a good investment of time and money.

    Let's assume that I get an e-mail from the program, and I send a copy to my boss and a purchasing agent. Then, I get for e-mail reminders, and so I am encouraged to follow up with my boss and with purchasing to ask them to accept your offer. They will most likely reply to the first e-mail, not the nurturing e-mails. If you use the same tracking codes for your control and test groups, you would have no way to find this out, and you would not be able to measure the full effect of your nurturing program.
  • Posted by Dane Robbins on 10/11/2012 at 5:57 PM Author
    telemoxie, thanks for your response. That is a valid point you bring up. I should clarify my initial question, however. This campaign is B2C, but we are using the lead nurturing label because it technically is a sales oriented lead that is contacted by a tour consultant and convinced to buy a travel destination. In this particular instance, we usually just have a retired husband and wife that makes the final decision. But to your original point, there is entirely a chance that a lead will purchase directly from the initial e-mail even after getting follow-up messages because they could have printed the original e-mail to make notes and reference. If the control and test shared the same ad source, you're right, we'd never know which group gets the booking.
BACK TO TOP

Post a Comment

Related Discussions

Community Info

Top 25 Experts
(Research/Metrics)