Inadequate Equilibria: Where and How Civilizations Get Stuck

Robin Hanson has old Overcoming Bias blog posts on that untaken, low-hanging fruit.

Then why don’t you just walk up to the decision-maker and tell them about the bias? Because they wouldn’t have any way of knowing to trust you rather than the other five hundred people trying to influence their decisions? Well, in that case, you’re holding information that they can’t learn from you! So that’s an “asymmetric information problem,” in much the same way that it’s an asymmetric information problem when you’re trying to sell a used car and you know it doesn’t have any mechanical problems, but you have no way of reliably conveying this knowledge to the buyer because for all they know you could be lying.

Cases where the decision lies in the hands of people who would gain little personally, or lose out personally, if they did what was necessary to help someone else;

Cases where decision-makers can’t reliably learn the information they need to make decisions, even though someone else has that information; and

Systems that are broken in multiple places so that no one actor can make them better, even though, in principle, some magically coordinated action could move to a new stable state.

If nobody can detect a cognitive bias in particular cases, then from our perspective we can’t really call it a “civilizational inadequacy” or “failure to pluck a low-hanging fruit.” We shouldn’t even be able to see it ourselves. So, on the contrary, let’s suppose that you and some other people can indeed detect a cognitive bias that’s screwing up civilizational decisionmaking.

I could be unusually right about macroeconomics compared to the PhD-bearing professionals at the Bank of Japan, but that weirdly low-hanging epistemic fruit wasn’t a low-hanging financial fruit; I couldn’t use the excess knowledge to easily get excess money deliverable the next day.

Where reward doesn’t follow success, or where not everyone can individually pick up the reward, institutions and countries and whole civilizations can fail at what is usually imagined to be their tasks. And then it is very much easier to do better in some dimensions than to profit in others.

Empirically, lots of people want money and acclaim, and base their short- and long-term career decisions around its pursuit; so achieving it in unusually large quantities shouldn’t be as simple as having one bright idea. But there aren’t large groups of competent people visibly organizing their day-to-day lives around producing outside-the-box new lightbox alternatives with the same intensity we can observe people organizing their lives around paying the bills, winning prestige or the acclaim of peers, etc.

To modify an old aphorism: usually, when things suck, it’s because they suck in a way that’s a Nash equilibrium.

Efficiency: “Microsoft’s stock price is neither too low nor too high, relative to anything you can possibly know about Microsoft’s stock price.”

Inexploitability: “Some houses and housing markets are overpriced, but you can’t make a profit by short-selling them, and you’re unlikely to find any substantially underpriced houses—the market as a whole isn’t rational, but it contains participants who have money and understand housing markets as well as you do.”

Adequacy: “Okay, the medical sector is a wildly crazy place where different interventions have orders-of-magnitude differences in cost-effectiveness, but at least there’s no well-known but unused way to save ten thousand lives for just ten dollars each, right? Somebody would have picked up on it! Right?!

The price is too high and will predictably decline, relative to public information, but there’s no way you can make a profit on knowing that.