Reliably Bad Reliable Sources
Bad Numbers from Prestigious Orgs
The Cleveland Fed’s Inflation Nowcast and the Chicago Mercantile Exchanges’s Fed Watch Tool are two of the most cited data-products in all of financial journalism. If you read financial journalism or track financial topics on Twitter, they are omnipresent sources.
You should trust neither. Both are obviously methodologically flawed and clearly inaccurate.
Their failures - and the failures of journalists to interrogate them - suggest serious problems with the information ecosystem around economics and finance.
Cleveland Fed Nowcast
The Cleveland Fed Nowcast makes predictions about the next inflation report. For example, this month, the Nowcast says that Core CPI M/M in August 2023 will be 0.38%. Over the past few months, here is a list of publications that have referenced the Nowcast as an authoritative source for inflation information:
When I started investing more seriously last year, I noticed this model was not very accurate and was often at odds with market-based pricing of inflation (on the losing side). This was particularly true for core inflation, the type of inflation that the Federal Reserve actually cares about. So I read The Nowcast Paper and slogged through the math and economics references to figure out why they were wrong. Here’s why:
In the absence of disaggregate information, we rely on the spirit of Atkeson and Ohanian (2001), who find that inflation over the previous four quarters is a difficult forecasting benchmark model to beat: assuming data through month t−1 are available, we forecast monthly core inflation using recursive 12-month moving averages.
This is a fancy way of saying “we gave up and a previous paper said that it was okay to give up.” Their prediction of core CPI inflation this month is a weighted average of what inflation has been over the last twelve months. That’s the model.
How insular a field economics must be for that to be an acceptable answer to nowcasting core inflation! There are free data sources about how prices are changing right now and large markets on inflation:
Consider these data sources about price changes in August:
You can get airline prices via Google. For example, earlier this week, I tracked fifty popular flight routes in August for about ~1 hour and found prices were down roughly 5% (not seasonally adjusted).
You can get wholesale used car prices from Manheim or the Black Book. These prices feed into consumer prices with some lag. These vendors will actually give you the information for free, and it explains a lot of the variance in core inflation, given the enormous volatility of used car prices.
You can pull prices from all sorts of large national vendors via APIs.
You can get rental price indices from Zillow or Redfin.
There are markets in which institutions bet large amounts of money on inflation via “swaps”. They probably have some views that are not equivalent to "what’s it been lately”.
There are instruments called “inflation-linked bonds” which pay a coupon which is based on the inflation rate from the consumer price index. As such, inflation-expectations can be derived from that instrument.
More recently, individuals can bet on inflation via prediction markets like Kalshi.com, and those forecasts are extremely accurate.
A true subject matter expert and hero of this blog - Omair Sharif - graces us with his wisdom about inflation every few months in podcast appearances and public writings. His Twitter ; His Website.1
Over the last three months, here is the track record of the Nowcast vs. looking at the final price on Kalshi:
July 2023: 0.40% prediction, 0.16% actual. Simple market-based forecast: 0.15%.
June 2023: 0.44% prediction, 0.16% actual. Simple market-based forecast: 0.20%.
May 2023: 0.45% prediction. 0.43% actual. Simple market-based forecast: 0.36%.
And for this month, Kalshi prices at 0.25%. Cleveland Fed is at 0.38%. If you believe the Cleveland Fed, betters are willing to give you 9:1 odds.
At this point, I should note that the authors of the Cleveland Fed paper are obviously very smart and their choice to work in public service speaks well of them as people. They produced a clear paper with a replicable methodology and document their error rates on a public website. In a world full of non-replicability and data manipulation, these are definitely the good guys. But the whole enterprise lacks grit, curiosity and common sense. The public deserves a better model from its Central Bank.
The CME Fedwatch Tool
The Chicago Mercantile Exchange is the largest options and futures exchange in the world. Over the exchange, the largest financial institutions in the world have options positions on perhaps the most fundamental market: “Will the Federal Reserve raise interest rates?” You might think that the largest options exchange in the world, reporting on the most important options market(s) in the world, would give you correct probabilities as reflected by options pricing. You would be very mistaken. It’s a free product and this is the Chicago Mercantile Exchange.
Instead of starting with the methodology, let’s proceed via what I’ll call a “proof by contradiction/common sense”.
Do you think that there is a 0% probability that the Fed will raise rates by at least 0.5% by next November?
Do you think that there is a 0% probability that the Fed will cut rates by at least 1.75% by next September?
According to the CME’s tool, these are the odds - rounds to impossible. But that’s odd because both of those things are entirely possible outcomes, not even that far from the modal outcome. If you track interest rates, you probably know that. On a trading platform like Interactive Brokers or TDAmeritrade, you can go look up the options pricing right now for an instrument like ZQ (Fed Funds Futures) or SOFR (Overnight bank rates) and you will observe the probabilities of these events are both well above 0%.
To be clear, the CME does have a methodology page which clearly outlines what they are doing, but what they are doing is a kind of alchemy, not worth delving into.
Why does the CME not just reveal options pricing? Perhaps there are legal constraints and this is truly the best they can do, perhaps they want to sell you the better data, perhaps they just assume no one cares about accuracy and the illusion of due diligence will suffice.
Despite the fact that these probabilities obviously contradict markets pricing, they are frequently cited by journalists whose job it is to cover the Fed. Examples:
Trusting the Experts
At this point, I hope I have convinced you that two frequently cited, highly prestigious economics sources are obviously bad and inappropriately cited by journalists. Does it matter?
This is a blog about betting markets, but the secret purpose of this blog is to chronicle the rise of a kind of new authoritarian and anti-intellectual political worldview, with betting markets deployed as an early warnings system. All across the world, from New Delhi to Jerusalem to Budapest to West Palm Beach, rising “populist” autocrats rail against journalists, and academics and all institutions of educated people.
The response from the left has been to dig in its heels - Trust the Experts, Trust the Science. And the danger of that response is that it favors a credentialism that alienates most people. It also is self-undermining when the experts turn out to be lazy or have incentives that do not align with truth.
The truth is that believing true things requires continuous work. You have to read multiple sources. You have to interrogate the track records of the experts. You have to question the incentives of the experts. In short, you have to think.
A reason for hope is that most people display intellectual grit like this all the time. Tens of millions of Americans think like this every year when they construct fantasy football lineups, or plan how to get better at Call of Duty, or predict the exact day and theme of Taylor Swift’s new album. None of this requires genius or fancy credentials. As with anything, it’s mostly passion and grit.
This man is the Nate Silver of the CPI - the first person you should read.
Personally, I don't use the CME Fedwatch to assess the full probability distribution, which (agreeing with your analysis) appears to assign too low probabilities to extreme outcomes. I do use it to gauge the median expectations implied the market. And I think it does an ok job at that - it generally accords with short-term bond yields after all.