Business intelligence analysts, marketers, and analytics mavens use many tools and techniques to extract useful information from all the meaningful data people generate every moment in every aspect of life.
I work with data every day, and I get great satisfaction from mining data to make things (like our clients' marketing programs) better.
Not everyone needs or wants to be an expert, however, and building expertise in data mining can be a struggle. Experts often confuse more than enlighten. But everyone should have an idea of what data analysts do. That knowledge can help you understand what data mining can and can't do.
Data mining is a science. That means we can repeat our successful analyses and use the results to become more effective and efficient at meeting our goals. All we really need is the recipe.
In the real world of business, the theory is not as important as getting accurate results. Data mining can resolve many of the questions and problems that arise in your daily marketing challenges.
A Cookbook Approach to Data Mining
Using a cookbook approach, even relative beginners can identify solutions by following step-by-step instructions.
Sometimes, the recipes can be very complex. For example, a direct marketer would be very interested in knowing how to find and address the right number of suspects or prospects for a special offer to optimize conversion rate and ROI. That's what our agency did for Volkswagen in China, the US, and Europe. We made a recipe that included all the data we collect—including behavioral, transactional, and self-reported data from social sites, from marketing campaigns and, with VW owners, even from their cars.
With a deep analysis of consumer profiles, we had the opportunity to create predictive models and move leads from "brand engagers" to "car-buying intenders."
For one marketing program for this client, we were able to use a data-mining recipe to identify people who had an 85%-95% probability of buying. We sent this relatively small group a generous offer voucher to entice them to buy a Volkswagen, and more than half responded and bought a new car using the voucher.
We can use another recipe if, for example, a client wants to find customers most likely to churn. This is an important target for publishers, finance, insurance, software, online services, and companies with long-term contracts with clients, such as telecommunications organizations. It's in these companies' best interest to identify customers who are likely to leave and try to work out ways to prevent it. In this case, the recipe is simpler.
You can use recipes to understand:
- How to find customers with the highest affinity for a particular offer (a discount or a new product, for example)
- How to find which customers to eliminate from a direct solicitation
- How to find the percentage of customers with the highest affinity to sign a long-term contract (such as a subscription or a cell phone contract)
- How to find the optimal number of communications to activate one customer
- How to find the optimal communication mix to activate one customer
- How to find and describe groups of customers with similar behavior patterns (to help you find new target groups)
- How to predict the future lifetime value of a customer
- And many more marketing and business issues
Data mining recipes discuss ingredients, instructions for preparation, and the (potential) fully baked results.
A Look at How We Create a Recipe
Here is an outline of the structure we use.
Challenge: The basic title of the recipe. For food, it might be "coq au vin." For data mining, it could be "How to Find Customers Who Will Potentially Churn."
Recipe ingredients include...
- Necessary data: All the data vital for this analysis. The data must have some direct relationship to the customer (e.g., marketing activities) or have come directly from the customer (e.g., purchase behavior).
- Population: The defined group we want to study. For example, when analyzing customers, we need to consider the definition of "active customer" for highly seasonal purchases and include purchasers from at least one complete cycle.
- Target variable: What is the data point that indicates the behavior we want to examine? This could be a binary value, such as "buying" or "not buying," or it could be a quantity, such as the dollar value of sales.
- Input data (must-haves and nice-to-haves): Key variables on which the analysis depends
- Dating mining methods: Do you broil or roast, simmer or sauté? In data mining, you may use one of any number of methods, such as logistic or linear regression, decision trees or neural nets. A good recipe will let you know the best approach—even when other methods might do.
- Data preparation: Getting your data ready for analysis. This is the mise en place of your data-mining recipe.
- Business issues: Defining the needed outcome of your recipe (how the results will be used, for example, whether in a strategic overview or in detail for segmentation purposes). It's important to discuss your definitions with colleagues and stakeholders who will be using the results.
- Transformation: How you make final models more robust. Mistakes can occur in any stage of data input or transfer. Just as you might beat a batter until lumps disappear, we need to replace missing values, exclude outliers (or ameliorate them) and smooth the data. Transformations can also summarize data in a useful way.
The step-by-step analytics part of the recipe includes...
- Partitioning the data: If there is enough data, you may want to split the data into training and test samples so you can best validate your model.
- Pre-analytics: This may involve screening out some variables (for example, variables that are all one value and depend on each other).
- Model building: The best-fit formulas or membership rules, for example, in cluster analysis
- Evaluation and validation: Learning how well the analytical process has performed in terms of value to the business and usefulness in decision-making, as well as how well the model fits the data. This usually involves applying the model to different subsets of the data and comparing the results.
- Implementation: Now that you have a good indication of the answer to your original challenge, what do you do next? The strategies you introduce will fulfill the value of all this information. You've pulled that coq au vin out of the oven, and now it's time to enjoy it.
These powerful data-mining techniques can bring enormous benefits by helping correctly pinpoint problems or opportunities. The strategies you derive from the information are even more important—and, with the help of proper data mining—will be built on a solid foundation.
