The thing that truly terrifies me about the election is that it reveals that our best methods of understanding and predicting reality are fundamentally inadequate. Brexit was the first hint, and this is the confirmation.
Trump's victory was not due to random chance, or weather, or some last-minute surprise. Nothing really changed in the last week. The depth of his popular support was a fact of reality days before the election, and a competent information-processing system would have learned of it.
It is a known fact among social scientists that polling always underestimates support for things that are seen as violating social norms. I assumed that the polls or models corrected for this somehow. They did not.
The polling was wrong. All of it. The experts were wrong. All of them. (Trump supporters predicting victory don't count, because partisans always believe they will win. And a few people online don't count, either, because you can find any claim online.) But what truly scares me is that the markets were wrong. All of them.
Before the election, I knew of the polling bias issue and worried that Trump had more popular support than the polls showed. But I did not place any bets, because I assumed that markets had already priced in this information. They had not. In the early afternoon of election day, election prediction markets had Trump at about 25%, and the stock and foreign exchange markets were assuming a Clinton victory with high probability.
Hedge funds spent millions of dollars on private polling and models to gain information that would allow them to beat the stock and forex markets. This is exactly how markets are supposed to reveal information. They give people billion-dollar incentives to obtain information, and then when people trade on that information, the price moves to reflect the private knowledge.
But even with the best performance incentives known to humanity, and all of the tools of social science and modern technology at their disposal, the hedge funds failed to obtain a basic fact about reality. If we cannot obtain the truth about people's feelings in a situation where the stakes are this high and information is so readily available, how can we hope to predict how our actions and policies will affect people's subjective quality of life?
