"It's hard to get analysis right," says Gary Angel in a post at the SemAngel blog. "[And] even when you do, it's hard to get it consistently right." You increase your chances, he argues, with a good process that prevents common pitfalls like these:

• Failing to establish measurements of success before deployment. "[I]t's always possible to find some evidence of success when you are allowed to choose the measure of success after the fact," he notes.
• Allowing vendors to measure their own performance. It isn't an issue of questioning a vendor's honesty; rather, says Angel, any self-interested party is naturally inclined to read data in the most positive—and, likely, least helpful—light.
• Conducting analysis without the aid of a professional statistician. According to Angel, this doesn't mean you need an entire team. "But if you have a team generating reporting and analysis on a regular basis," he says, "you need at least one gate-keeper reviewing it and quashing the most abusive practices."
• Neglecting to tell an analyst that something has changed. If your technology and marketing managers don't coordinate with those who study your data, critical insights might be lost.

The Po!nt: "Good process is very much about protecting ourselves from the things that cause mistakes so that we have a chance to be consistently correct," says Angel.

Source: SemAngel. Click here for the full post.

→ end article preview
Read the Full Article