Know-How Exchange : Strategy

Like you, my experience with modeling to measure the effectiveness of promotion activities, and allocate among marketing mix elements, is heavily slanted toward consumer packaged goods. But I have a few experiences in other categories -- not including high-tech/telecom.

Nonetheless, I find it hard to imagine that the basic principles would not apply, especially if you have granular data -- by SKU, by store/geography, and by day. You may have to build the basic model based on some aggregates (to eliminate wild unexplained variances or "flukes"), but once the baselines are established, there is absolutely no reason to think the analytical approach won't work.

Sorry I can't give you a hard example, but your instinct is consistent with mine ... so I'm inclined to trust us on this one.

Is it possible that the total number of observations is too small for reliable statistical analysis? When you look at a single SKU, for example, if there are only 2 or 3 sales a week, maybe the limitation on the analysis is really a paucity of observations.

In large markets and high-volume product categories this isn't a problem, and we just assume it will work everywhere. But if you're in a relatively small market, and if volume is relatively low, maybe the methodology is less effective, or at least not so robust as our gut instincts would indicate (based on our experience in a different marketplace environment).

Turkey is about 1/4 the size of the United States in population, so it shouldn't be a big problem, but maybe the experience there is in just one city or region, and hence their skepticism.

When an assumption to the model changes, it may cause the projections to shift. This is true not only within a segment, but also in the culture itself (a financial crisis, a war, etc.). Without any modeling, the company is doing things based on "gut feeling". Can you get their past data, and have them indicate where they saw the inflection points/paradigm shifts? Can you show them that a single model is actually made up of sub-models and that some sub-models are unaffected by shift (i.e., relative company ranking, sales to corporations, etc.) or shift more slowly relative to a "major event"?

Also, you've probably read "The Black Swan" by Nassim Taleb - but it may be worth sharing the book with the mobile phone service people as well. It's all about events you can't predict and its impact on business.

Koen:
I'm currently in the process of reviewing Marketing Mix Modeling / Optimization (MMM) suppliers for a relatively high-tech product with long purchase cycles, high ticket price, etc. All the suppliers I've reviewed have experience in dealing with similarly-characterized products. In addition, there are a host of major changes taking place in the business model which could present challenges to the modeling so I had similar concerns about the validity of the technique.

To generalize, MMM comes down to the cadence, length of history, and quality of the data inputs and of the model itself. Even if some of the inputs (like free services, distribution, product features, pricing, etc.) change, there is still value in understanding the contribution that marketing components make to sales.

I'd expect this to be especially true if the most of the industry follows the innovator (i.e., everyone matches the free service) - now everyone has a relatively equal footing again and has to determine "what else drives the business?" Even if they don’t, haven’t all the competitors had various features, services, attributes and benefits that they need to communicate? If so, marketing should still have an impact on purchaser response: Consumer preferences, and what you’re talking about, may change, but you still need to use marketing to communicate it and determine the best channels to reach people.

Is another possibility modeling the impact of other changes/innovations/launches over time for the client and the competition? There may be a possibility to use econometric modeling to learn from history as to what works (spend levels, channels, etc.).

In any case, what’s the alternative? Using no data to make the decision? Why not propose using the modeling, and other in-market testing, to reassess the ROI landscape in this brave new world of changing technology and offerings? There should be some validity to the overall/underlying models (and you can test the hypothesis of how predictive MMM is) plus you can determine what works going forward.

Good Luck
-Steve

Dear Koen:
You might consider introducing old-school direct marketing measures: treat each activity as a campaign with its own unique inputs, criteria, and ROI measures; track life-time-value by small highly targeted segments, etc. Your client/colleague might find this a more digestible concept that they can sell to management and still feel good about keeping their jobs.

I spent many years marketing telecommunications in the US with the then ever changing US Telecommunications Act of 1996 (TA96). The law was very vague and we often didn’t know what activities might be legal from one day to the next. Legal action and court rulings dictated our daily business actions, and we would even occasionally act to provoke legal action from our customers to force court clarification on specific items.

The implementation of direct marketing measures was a saving grace for our department. We were able to report results to corporate management in a meaningful manner, we forced ourselves to be incredibly nimble, and we felt more able to be proactive: if the law or market changed, we cancelled a campaign and launched another.

Best regards, Margaret

Oh - and data availability restrictions simply don't occur in telecoms. They have access to the last 5 years of revenues account by account and they have access to the cost and therefore the profit on each and every call - pre-paid or contract.

The data-set will be in the billions if you include texts

If the data looks Granular, you are using your telescope the wrong way round!

Steve Alker
Xspirt

Hi Koen

Here's the result of my initial observations to arun in his personal email which he had to do if Koen and I were going to see any Graphs! Are you listening MP Management - if not we'll do it ourselves!!

Like you I respectfully disagree. That's exactly what I put into the difficult but calculable area of MMM as long as you look at options, caveats, nodes and what-if and all that crap.

Arun

Here’s a few notes – probably more later!

You say that sales fall off to the pre-promotion level after the “reverberations” bit. You figures and your graph don’t reflect this, they reflect linear growth.

Also you talk of engineering services – that’s not telecom sales which can be B2B but is in the main B2C

In B2C and in Telecoms to message will go Viral and WOM. You won’t get that in engineering services! Don’t make dodgy assumptions

By the way, out of courtesy to MarketingProfs members and Koen in particular, this should be posted as it is on the KHE. I too think that it’s sad that they don’t have a place for equations and graphs. I have assumed that you wonldn’t object are there are no secrets in your email and my reply refers to graphs no-one can see. Can you host the graphs on a website and put in a lik to the discussion on KHE? Or shall I do it?

I have made some convenient assumptions: - No some of them appear to be very inconvenient if you want an answer or answers you can work with!-Sorry, couldn’t resist that one!

1. There is only one promotional effort and nothing else is happening in the market:- Really? That’s almost unique in marketing. Something else will have been promoted which made a demand on buyer’s disposable cash. An iPad is not a mobile phone, but if a punter is buying one of those, he is unlikely to buy an iPhone in the same month unless he’s loaded. I used to battle to persuade my sales teams that the competitors products were not the only competition. The MD really had to both Want and Need our Electronics Kit rather than toddling off and treating himself to a Ferrari instead and stuff the purchase of a £70,000 for a month or a year!

2. Efforts of competitors do not affect us:- I realise that you are operating in a different market, but if you have competition who are effective – not just who spend a lot, this statement is bats. Of course they effect you, you just can’t measure it in the noise
3. The Promotion is introduced in the month of April:- Obviously not on April 1 (April fools day in the UK!)

4. I follows an exponential growth path (Which may be different for different promotions depending on how good the effort is):- It can be any old shape it decides that it’s going to be depending on the planning and what happens on the ground. Here it is has an exponential phase when, two months after the promotion, the sales enter the viral and WOM stage. That’s roughly from June to November) You would expect that with viral things. Most promorions are sort of bell shaped with, if you are lucky a long tail which is exploitable

5. There is a second spike due to word of mouth about the promotions (reverberations) which is inevitable in a market that requires lesser involvement in the decision making process and is dependant on testimonials, I do not know if telecom is such a market. i am just assuming this for a wors case scenario:- What second spike? I can’t see one in any of your graphs or figures. Please clarify. If you differentiate the graph you kind of get two spikes, but that is rate growth of sales growth – not rate of sales. I’m baffled

6. Sustains for a period of2 months and begins to decline:- No, it gets exciting for 2 months and then it becomes linear growth – look, I think that the problem here is that you are not sorting out the promotional sales from the organic growth in sales from all sales and marketing activity since the company was formed. If you want figures for the promotion alone, you need to work this out.

7. Since there are no other influencing factors, the sales return to pre-promotion levels:- No they don’t, they continue to grow but are linear. Why are you applying the first differential of your graphs to all your statement?

In summary I think that your problems as such are of not identifying the sales produced by past effort and that the true picture of the sales attributable to the promotion are therefore hard to interpret. Nothing in this negates the worth of MMM in Koen’s question.

If your business is CAD sales or consulting then making a comparison with telecoms for the sake of it is to my mind silly. Your customers will be all B2B and from the sprinkling of CAD customers on my Forecasting CRM system depending on how big the CAD solution is, they work on 50-200 sales a year.

So why are we looking at graphs of telecoms sales if your product is CAD sales or have I got the wrong end of the stick

Look forward to hearing from you

Steve Alker
Xspirt
________________________________________
From: Arunkumar [mailto:arunkumar@cads.ae]
Sent: Wednesday, April 14, 2010 5:11 PM
To: steven.alker@btopenworld.com
Cc: koen.h.pauwels@dartmouth.edu
Subject: Reg. Can Models Guide Allocation Strategy In Hi-tech ?

Hi Guys,
I did not ask you if it was ok to write to your email IDs, but have taken the liberty to already. So if you do not want mails regarding the posts then please reply so, so that I can refrain from doing so in the future. I just could not explain without the diagram. Voice could have done better, but pictures are good too. So here goes:

If you look at the picture you will be able to see that the promotion has really started paying off only in the month of August and has reached the peak of its deliverability in the month of November, though the promotion was started in the month of April. I do not really know the actual time it takes in the telecom market. But this is a very conservative estimate in the engineering services market. Actually it can take 3 to 5 times longer than this depending on the efficiency of the promotional effort. So now to estimate the short term response of the promotional effort using projection of time series data will not be correct unless we know exactly when the effort is going to really start paying off and how long it is going to last. One may say that based on past data, we can identify an approximate time for this to happen. But we must bear in mind that it was not the only influencing factor then and it will not be the only influencing factor now. But on a long term you can judge the approximate effect this will have in the combined influence of other factors.

But like all projection methods, whether you use it or not depends on what is at stake. The data is not definitely going to be accurate, but if the tolerance is high, then you can still use this method for short term projections. However, if you want accurate information, then I still do not feel that this is the best method to do this.

Regards,
R. Arun Kumar,
Manager - Marketing.

Koen

Of course the future is going to be different so have your what-ifs and you models ready and don't go for any iterative non-linear crap (see my posting which is coming up for closing time) if you are doing long term from a rapidly changing market which is therefore non-linear.

For heavens sake, if the market changes quickly and they have to build a model in a hurry, we are only likely to charge an emergency service consulting fee because without us (You in this case) they are stuffed and it is very hard work and hard to get right in a short time.

Do these guys believe in random change and pre-destiny at the same time?

Its all contradictions. I’m not sure how the teachings of Prof Stephen Hawkins go down in a Secular Islamic State, but he once said "Even the most devout determinist I know looks before they cross a buy road"

This is nuts!

Steve Alker
Xspirt