Monday, October 25, 2010

Predicting People

This article on crime is interesting by itself, but it also leads to some deeper thoughts:

a team of researchers from the University of California, Los Angeles, adapted a computer program used by seismologists to calculate the likelihood of aftershocks. They then seeded it with actual LAPD data on 2,803 residential burglaries that occurred in an 18km-by-18km region of the San Fernando valley, one of the city's largest districts, during 2004. Using the seismological algorithms, the computer calculated which city blocks were likely to experience the highest number of burglaries the next day, and thus which 5% of homes within the area were at particular risk of being broken into.

We see more and more of this kind of thing all the time.  Computer programs and mathematical algorithms can do a very good job of predicting human behavior.  But most people do not have a real understanding of what this implies.  In this example, the behavior of humans was accurately predicted using equations designed for predicting the movement of rocks.  Think about that for a moment.

When human action can be calculated so accurately, as if we were just a collection of mindless natural objects, what does that say about self-determination and free will?

This problem has been around for over a century, starting when sociologists first started analyzing large data sets on human behavior.  The scientists of the time were shaken to their cores when they realized just how strong and robust the correlations were between things like childhood poverty and later criminality.

All of our social intuitions tell us to treat people as unique individuals, and to assume that their actions are determined entirely by their character traits and attitudes.  This makes sense when the world you are aware of only has a few hundred people and they all grew up in about the same situation.  But when there are millions of people, and they live in vastly different physical, social, and institutional environments, cold unfeeling math is far more useful at predicting what people will do.  At that scale of analysis, everything that humans instinctively care about and think of as important to predicting people's actions is reduced to meaningless statistical noise.

The impression I get is that people are a lot like gas molecules.  Physicists and chemists know that the future motion of a single gas molecule is completely impossible to predict.  Despite the fact that its movements are governed by very simple physical laws, you cannot know where it will end up after about 20 collisions with other gas molecules.  A tiny uncertainty in initial measurement will make a vast difference in outcomes, and the Heisenberg Uncertainty Principle guarantees that there will always be that tiny measurement error in it initial state.

However, the aggregate behavior of millions of gas molecules is very easy to predict in most situations.  A few simple equations involving temperature and pressure will tell you everything you need to know about what a gas will do in response to physical changes.  Things get a little more complex when chemical changes are involved, but with the right equations and data sources you can predict with extraordinary accuracy exactly what will happen when a gas comes in contact with just about anything.

Similarly, the behavior of a single human being is almost impossible to predict, but the aggregate behavior of large numbers of people follows very steady and predictable mathematical equations.  The equations can get a lot more complicated than the ones involving gas molecules, but they can still be learned, known, and applied usefully.

Of course, these equations will do nothing to help you navigate your relationships with the people in your life.  Your instincts are still valid for that.  But if you want to make smart decisions about running a country or economy or large business, you need to use the math that treats people as objects.

This means that the intellectual distance between powerful decision makers and ordinary people is likely to grow.  Even aside from the inherent fact that power corrupts, it is hard to retain your humanity when you must analyze people statistically.

No comments: