You have been predicted.
Companies, government, universities, law enforcement. All are using computers to predict what you will do.
Will you click on the link in the email?
When will you die?
Will you pay your credit card bill on time?
Are you pregnant?
Dr. Eric Siegel recently released Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie or Die. It’s a fascinating book that has surprisingly broad ramifications for all of us. Eric is a former Columbia University professor, the founder of Predictive Analytics World and Executive Editor of the Predictive Analytics Times.
Let’s start with the definition. What is predictive analytics?
It’s technology that gives organizations the power not only to predict the future, but to influence it. The shortest definition of predictive analytics is my book’s subtitle, the power to predict who will click, buy, lie, or die. Predictive analytics is the technology that learns from data to make predictions about what each individual will do–from thriving and donating to stealing and crashing your car. By doing so, organizations boost the success of marketing, auditing, law-enforcing, medically treating, educating, and even running a political campaign for president.
Why should the average person care about predictive analytics?
Prediction is the key to driving improved decisions, guiding millions of per-person actions. For healthcare, this saves lives. For law enforcement, it fights crime. For business, it decreases risk, lowers cost, improves customer service, and decreases unwanted postal mail and spam. It was a contributing factor to the reelection of the U.S. president.
Let’s jump to politics then. How did President Obama’s campaign gain an edge by using persuasion modeling?
The Obama campaign’s analytics team applied persuasion modeling (aka uplift modeling) in the same way it can be applied to marketing: drive per-person (voter/customer) campaign decisions by way of per-person predictions. If an individual is predicted to be persuadable, then make campaign contact (e.g., a knock on the door). By utilizing resources (campaign volunteers) more effectively in this way, the campaign enacted the new science of mass persuasion. They proved this won them more votes, within swing states and elsewhere.
Everyone is talking about “big data” but data on its own isn’t interesting or useful. You explain how data can show incredibly interesting insights including the fact that if you retire early, your life expectancy drops. Tell me more about that and what else we’ve learned from it.
Beyond the great hype around so much data, the real question is what to do with it. Answer: use data to predict human behavior.
The whole point of data is to learn from it to predict. Talking about how much data there is misses this point. What is the value, the function, the purpose? The one thing that makes the biggest difference to improve how organizations operate is to predict.
Data is the encoding of experience. It always contains predictive insights such as the retirement example you mention. Predictive models are derived automatically by learning from data how best to combine the factors about each individual to render predictions for that individual.
In predictive modeling, technology predicts an individual’s behavior by leveraging the ‘wisdom of the crowd.’ Explain how you use collective intelligence in this way.
Since we need to produce millions of per-person predictions, we can’t use the wisdom of a crowd for each one – not in the traditional sense. Instead, leading technology called an ensemble model employs a “crowd of models” – just like a group of people who compensate for one another’s limitations, so too does a group of analytical methods.
These individual predictions also raise significant privacy concerns. We must touch on Target and what happened when they used data to predict customers who were pregnant. You were squarely in the middle of that. Tell us what happened and what you learned.
With the advent of predictive analytics, organizations gain power by predicting potent yet–in some cases–sensitive insights about individuals. The fact is, predictive technology reveals a future often considered private. These predictions are derived from existing data, almost as if creating new information out of thin air. Examples include Hewlett-Packard inferring an employee’s intent to resign, retailer Target deducing a customer’s pregnancy, and law enforcement in Oregon and Pennsylvania foretelling a convict’s future repeat offense.
In my opinion, what happened with Target was they were fairly cautious about, and aware of, PR issues around that endeavor — in comparison to many technologists on such projects — but they simply weren’t cautious enough. An understandable mistake as this was a first-of-its-kind collision between this kind of technological feat and the public’s response. If a marketer derives medically sensitive info, they need to present themselves as aware and secure just the same as a healthcare institution. A lack of PR savvy is more than just cosmetic; it could indicate internal practices lack the proper level of care.
In one chapter, you indicate “correlation does not imply causation.” It’s easy to make a mistake when looking at things from only one perspective. Give me an example of this and why it’s something to watch out for.
Consider this: Increased ice cream sales correspond with increased shark attacks. Why do you think that is? A causal explanation could be that eating ice cream makes us taste better to sharks. But another explanation is that, rather than one being caused by the other, they are both caused by the same thing. On cold days, people eat less ice cream and also swim less; on warm days, they do the opposite.
In general, we detect predictively-valuable correlations in data, which themselves help us predict but do not tell us about causality, i.e., “how the world works.” Knowing about causality is a luxury we often don’t need.
One of the most popular Jeopardy events was watching IBM’s Watson beat humans on the show. What role did predictive modeling play in the human’s defeat?
This is one of the most inspiring accomplishments of predictive analytics. On Jeopardy!, the questions can be about most any topic, are intended for humans to answer, and can be complex grammatically. It turns out that predictive modeling is the way in which Watson succeeds in narrowing down the answer to each question: it predicts, “Is this candidate answer the correct answer to this question?” It knocks off one correct answer after another — incredible.
What’s the biggest misconception about your field?
There’s a tie between two big misconceptions:
1. Predictive analytics is the same as forecasting. But no — instead of foreseeing general trends like overall sales or the direction of the economy, predictive analytics renders a prediction for each individual. In that way it improves mass-scale operations, driving them one micro-decision at a time.
2. Predictive analytics is difficult to understand.
What’s the future of prediction?
Mostly, the future is here, it has arrived. But most people aren’t aware how much they’re being predicted and how this affects one’s experience every day, since the decision process behind transactions and interactions with companies and government agencies isn’t transparent; an org is a black box just as your friend’s decision-making brain is.
Traction is in full swing and more and more operations will be driven by prediction. We’ll be able to see this more visibly for certain applications like product recommendations (music, movies, books), driving directions that avoid predicted traffic, healthcare decisions that are more clearly demarked by predicted odds, and much more.
Author Bio: Eric Siegel, Ph.D., founder of Predictive Analytics World and Text Analytics World, and Executive Editor of the Predictive Analytics Times, makes the how and why of predictive analytics understandable and captivating. In addition to being the author of Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, Eric is a former Columbia University professor who used to sing to his students and a renowned speaker, educator and leader in the field.
