Is statistics the new plastics? And is it, as Hal Varian says, sexy?
Great article last week in the New York Times: For Today’s Graduate, Just One Word: Statistics.
I really liked the Hal Varian quote about statistics being sexy, but I have to point out one thing. The article doesn’t draw the fundamental distinction between people with extensive mathematical backgrounds and those who have been sitting in front of a statistical software console for the past couple of years.
Because there’s a huge difference between knowing why you’re doing a certain kind of analysis…and just plugging away with the tools at hand. Keep in mind that those tools have become powerful beyond many of our skill sets.
It used to be that you’d take a course in time series analysis. Or you’d spend a year studying stochastic processes. Or perhaps an entire advanced degree in Markov processes and queuing theory.
Today, these advanced degrees sit in “modules” that cost a few thousand dollars. Last winter, my previous employer didn’t have the tools to do time-series forecasting, so I found freeware that handled regression and time series quite well — ARIMA (1,1,0) as I recall.
But here’s the rub. This is mumbo jumbo that nobody understands. Name the MBA program where they cram in four years of advanced mathematical modeling. Most business people understand confidence intervals — maybe. But start talking about stochastic models, and you’ll lose just about everyone. You might as well spend your evenings transposing your checkbook into base 16 and balancing it that way…just for fun.
I learned a long time ago that we can figure out most things with great certainty — but it’s very hard to get people to make decisions based upon abstractions such as models and confidence intervals. Sure, there are businesses where the ant farmers and complexity scientists can have an impact, but those teams of people still need translators in order to work within their own companies. Data mining is the practical application of logistic regression to figure out how to sell more stuff to existing customers. That’s easily explained in most meetings.
But what next? Once we extract the additional yield from our current customer bases, we’re going to hit a point where we’re going to cycle from one noninferior market solution to another. That won’t be productive, and we’re going to wonder what happened?
In the end, as we’ve learned with the financial markets, there’s no substitution for growth. If statistics is a trend for the next decade, is it just another form of arbitrage? Is the perceived growth just another form of substitution? Or will the statisticians find something that just isn’t there today.
I, for one, am hoping that this trend will do more than find the $5 bill that got washed in my pants last week. I was going to find that one anyway.