Question

Topic: Research/Metrics

Statistical Validity Of Price-testing Results?

Posted by markp on 6/15/2009 at 8:32 PM ET 250 Points
Prior to launching a new product to a small, hard to reach, niche b2b market we have conducted some online price testing but are not feeling very confident that the results are sufficient to make anything more than a "gut-feel" decision. I've seen some similar posts and excellent responses here on the topics of sample sizing, results validity determination, etc but have to admit that with my currently jet-lagged brain I'm having trouble applying the lessons to my particular circumstances.

Printed and emailed pieces invited prospects to visit a site to learn more about the product, those visiting would see one of four price points (prices presented were evenly distributed and exclusive, each visitor seeing only that one same price on repeat visits).

Total number visiting the pages was around 1000 (distributed at 250 per price point). Number of orders at each price point ranged from 10 to 18.

How confident can I be that that 18-order price point is valid and scalable beyond this small test?

What is the proper statistical test to apply and is there an online calculator that would apply to this type of A/B/C/D test results?

Thanks,

Mark
To continue reading this question and the solution, sign up ... it's free!
SUBSCRIBE
Or, SIGN IN

RESPONSES

  • Posted by CarolBlaha on 6/15/2009 at 9:44 PM Member
    you are trying to apply a statistical test to a subjective audience.

    When I worked with a major pizza franchise/chain, we did a similar survey. We asked what about a fat free pizza? Results were staggering-- yes! Then we test released it, and nobody cared. It wasn't about taste-- they never tried it-- nobody cared.

    So people say what they believe is truth, but not necessarily how they purchase. As in the pizza example, we want to be fit and lean-- but when you go for pizza, we go for that warm fuzzy fat feeling. I haven't eaten a pizza in 15 years and if I want to stay in my size 2's, even with claims of fat free-- it won't be on my plate. I don't care how it tastes or if they give it away.

    Back up your stats with real competitive shopping. Honestly evaluate your product against your competitors. Do you have true measurable differences that compel a higher price point? Or do you have a true price advantage?

    Another analogy-- I am newlywed (at 55!!) and I met my husband on an internet dating site. Both of us had experiences with profiles of people how they visioned themselves to be-- not what they are at this moment in time. Same with surveys. They envision a product at their wanna be price point-- that does not mean its set in stone and you can't get more.

    People do NOT buy on price. They buy on the benefits that solves problems. I think your research so far is great-- expand on it for validity.

  • Posted by markp on 6/15/2009 at 10:10 PM Author
    Thanks for the rapid reply Carol. Just to clarify, these pages all sell the products hard with lots of WIIFMs and are fully e-commerce enabled and strongly ask for the order -- so to the customers the orders are real, not just survey replies or intentions.

    Like you we've found that how someone votes in a survey is not always how they vote with their wallet and real money!

    In this market the number of potential purchasers we can reach discretely in a price test is limited, and of course we are under some time constraints in determining the right price before we launch it into wider wholesale distribution.

    So I'm leaving to drive home to a salad for dinner but now I'll be thinking about a big, thick slice of cheesy pizza! =;-)

    Thanks again, Mark
  • Posted by CarolBlaha on 6/15/2009 at 10:21 PM Accepted
    So my comment sums up as-- follow the results and your gut. And remember- you can always drop a price --and your clients will win and call themselves tough negotiators-- but you'll never get more than what you list as your selling price.

    PS, I just finished my salad-- my new hubby ate a cheese ladin authentic mexican something not only full of cheese but enough salt that I am thinking of increasing his life insurance!
  • Posted by markp on 6/18/2009 at 8:30 PM Author
    Thanks Anna,
    I agree that the problem (turned out to be) too few responses spread across too many price points. When I set out I assumed more visitors would result from the various postal and email invites...

    The prices tested varied by $10 increments -- i.e., $49, $59, $69, $79. Great suggestion on the retest, but unfortunately I won't have time to test again so will probably decide to go with the 18, or the one one step higher, knowing that once into distribution it's much easier to come down than raise prices later. And as you say, I can always discount off or add value if I find I've picked the wrong one.

    Thanks very much for your helpful suggestions, I appreciate them!

    Mark
  • Posted by koen.h.pauwels on 6/19/2009 at 3:27 AM Accepted
    Hi Mark,

    If you do not have time to run another experiment, going with the 18 price point indeed makes lots of sense - it is easier to drop it later and you are not leaving money on the table by underpricing. Being a quant guy myself, I do have to say you are selling yourself short by calling this a 'gut-feel' decision: I am impressed by the excellent design of your pricing experiment, which is superior to much I have seen companies do....

    Cheers and good luck!

    Koen
BACK TO TOP

Post a Comment

Related Discussions

Community Info

Top 25 Experts
(Research/Metrics)