hayekian knowledge problems

recently, i came across the local knowledge problem, also known as the hayekian knowledge problem. it is the concept that the information required for economic planning is distributed among individual actors, as each has varying levels of personal knowledge, and hence exists outside of the knowledge of a single central authority. and this is a problem, because there is no incentive for people to share the information they hold with a central authority.

practically every individual has some advantage over all others because he possesses unique information of which beneficial use might be made, but of which use can be made only if the decisions depending on it are left to him or are made with his active cooperation

— hayek

this is likely why most western countries operate with market based economies, they have recognised that markets allow unique information to be priced into decisions, and hence allow for more optimal outcomes, especially in the private market. while reprehensible, insider trading is an example of the knowledge problem being solved. whilst the information itself isn't publicly available, it is available in the price of the asset, arguably making the market more efficient in regards to asset allocation. markets are a truth seeking discovery mechanism for revealed preferences, and allow the wisdom of the crowd to be priced in.

this trend doesn't end with private goods. economic related government decisions aren't entirely made in isolation from the market, bond vigilantes ensure that governments adopt prudent fiscal policy, whilst other leaders, such as trump, look to the stock markets' reaction to more controversial policies such as his liberation day tariffs. however, this is not an equitable solution to the knowledge problem, the bond market is restrictive for normal people and at best a weak beta to signal knowledge about government policy.

hayek argued a step further, suggesting some form of decentralisation, stating that "we need decentralisation because only thus can we ensure that the knowledge of the particular circumstances of time and place will be promptly used."

prediction markets

this brings us to prediction markets. what i believe to be an elegant and interesting solution, they allow people to wager on their beliefs and knowledge. they are elegant for a few reasons, firstly they provide an incentive for people to share the information they hold without divulging the information itself, it becomes priced into the odds of the market. secondly, they tend to be more accurate than polls, as they reflect what people are actually willing to stake on an outcome, and odds update in real time. they allow people to perform eminently useful functions based on special knowledge of circumstances of the fleeting moment not known to others. prediction markets also serve as a public good, allowing people to see real time odds for niche events, and in theory with all idiosyncratic information priced in.

whilst better than nothing, they do still run into the problem of liquidity constraints. someone might have very strong beliefs about an event, but may lack the capital needed to fully price the information they hold into odds on prediction markets, or speculators in the casino economy might be trading on prediction markets while lacking strong knowledge about the underlying market. however, whilst traded odds do not reflect someone's interpretation of fair odds, they will always have an incentive to help the market price in their local worldview.

and the nice thing about markets is that they're meritocratic, especially prediction markets. long term profitability on prediction markets implies a local worldview that is more aligned with reality than that of the market's dollar weighted average participant. this differs from normal asset markets, where profitable trading can just mean your local worldview is more aligned with the worldview of the majority, rather than reality, especially on more speculative assets.

but the further I studied markets the more I realized that you gain no sanity studying the deranged.

this gives rise to the popular epithet: would you rather be right or would you rather make money? your local worldview may correspond to that of reality more than the market's dollar weighted average participant, but if the average local worldview is sufficiently different to yours, you can be unprofitable. in traditional markets, it is possible for assets to remain disconnected from their fundamental value, or to have a fundamental value hard to price in accordance with your worldview.

hence, in traditional markets, the right thesis alone means nothing, the correct expression is just as important, if not more important, than the thesis itself.

the market can remain irrational longer than you can remain solvent

meanwhile, in prediction markets, the thesis is the expression, hence it becomes a lot easier to find relative "truth". however, prediction markets themselves do not solve the knowledge problem. whilst they allow for individual knowledge to be priced into the odds of an event happening in a meritocratic way, they do not cleanly resolve information dispersion related specifically to economic planning in a centralised economy. however, we can apply the same concept, bringing us to futarchy.

futarchy

futarchy is a form of government proposed by the economist robin hanson, where prediction markets are used to determine which policies will have the most positive effect on predefined measures of national wellbeing.

in theory, this seems very attractive: we have already established that prediction markets tend to be more truth seeking than normal asset markets, and that central economic planning can struggle due to knowledge dispersion, hence applying prediction markets to economic policy seems straightforward. however, some initial problems quickly become present.

firstly, it is difficult to find good, unbiased metrics to measure. GDP growth or GDP per capita are incomplete, they do not account for levels of inequality or any other broader, more granular factor. welfare is multi dimensional, it is hard to collapse this into one number, and the fact that a metric is being measured might change behaviour in itself. by trying to turn societal success into tightly predefined metrics could cause outcomes that are locally optimal but globally suboptimal.

when a measure becomes a target, it ceases to be a good measure

— charles goodheart

secondly, the incentive design of futarchy allows for manipulation. an oracle problem persists, the data being tracked may be inaccurately measured or may be influenced by corruption if the people involved with tracking it stand to benefit from an outcome.

manipulation could also include someone staking a large amount of money on a policy that is attempting to boost GDP, such as lower taxes for software companies, whilst being the CEO of a software company. even if the measure fails to boost GDP, it is entirely possible their company and indirectly they will earn more as a result of the policy than they lose for supporting a negative outcome.

after some consideration then, it becomes apparent that current versions of futarchy are unsuitable for use today, so whilst it appears prediction markets can solve the knowledge problem in certain areas, it remains unsolved as a matter of public policy.

i find the hayekian knowledge problem and its quasi solution in prediction markets aggregating local knowledge beautiful, but the evident flaws in futarchy do make me wonder if there is perhaps a hybrid version of it more suited to helping policymaking, or if policymakers can simply take guidance from knowledge aggregated through prediction market prices to help them make decisions.