Super Crunchers by Ian Ayres Read Free Book Online Page B

item on eBay. Then if you pay me ten dollars, I’ll guarantee it will go for at least a thousand dollars. And if it doesn’t, I’ll pay you the difference.” Of course, auction bidders will also be interested in these predictions. Bidcast software that will suggest whether you should bid now or wait for the next item is sure to be coming to a web portal near you.
    Sometimes Super Crunching is helping consumers just get through the day. Inrix’s “Dust Network” crunches data on the speed of a half million commercial vehicles to predict traffic jams. Today’s large commercial fleets of taxis and delivery vans are equipped with global positioning systems that in real time can relay information not just about their position but about how fast they’re going. Inrix combines this traffic-flow information with information about the weather, accidents, and even when schools and rock concerts are letting out, to provide instantaneous advice on the fastest way to get from point A to point B.
    Meanwhile, Ghani is working to use Super Crunching to personalize our shopping experience further. Soon, supermarkets may ask us to swipe our loyalty cards as we enter the store—at which point the store will data mine through our previous shopping trips and make a prediction of what foods we’re running out of. Ghani sees a day when the supermarket will become a food shopping advisor, telling us what we need to buy and offering special deals for the day’s shopping trip.
    The simple predictive power of a good data crunch can be applied to almost any activity where people do the same thing again and again. Super Crunching can be used to give one side an edge in a commercial transaction, but there’s no reason why it has to be the seller. As more and more data becomes increasingly available for free, consumer services like Farecast and Zillow will step forward and crunch it.
    In Regressions We Trust
    These services not only tell you which way the price is going to move, they also tell you how confident they are in their estimates. So with Farecast a consumer might learn not only that the fare is expected to drop, but also that this type of prediction turns out to be correct 80 percent of the time. Farecast knows that it doesn’t always have enough data to make a very precise prediction. Other times it does. So it lets you know not only its best guess, but how confident it is in that guess. Farecast not only tells you how confident it is, but it puts its money where its mouth is. For $10, it will provide you with “Fareguard” insurance—which guarantees that an offered airfare will remain valid for a week, or Farecast will make up the difference.
    This ability to report a confidence level in predictions underscores one of the most amazing things about the regression technique. The statistical regression not only produces a prediction, it also simultaneously reports how precisely it was able to predict. That’s right—a regression tells you how accurate the prediction is. Sometimes there are just not enough historical data to make a very precise estimate and the output of the regression technique tells you just this. Indeed, it gets even better, because the regression tells you not only the precision of the regression equation on the whole, it also tells you the precision with which it was able to estimate the impact of each individual term in the regression equation.
    So Wal-Mart learns three different kinds of things from its employment test regression. First, it learns how long a particular applicant is likely to stay on the job. Second, it learns how precisely it made this prediction. The predicted longevity of an applicant might be thirty months, but the regression will separately report the probability that the applicant would work less than fifteen months. If the thirty months prediction is fairly accurate, the probability that the applicant will work only fifteen