This is part 5 in a series on how we used LLMs to do reliable financial research. I wrote about earnings releases and 10-Ks. McKenna wrote about earnings calls and 10-Qs.
Management assessment is our spiciest category. Our previous articles covered how we got LLMs to not omit reliable and relevant financial information. This one is more about the findings.
You can think of this article as: if you get good at using LLMs to research financial filings, at scale, and with high reliability, what can you learn about the fortunes of public companies?
Our method here was:
- For each S&P 500 company, read a 10-K or two from 5 years ago. See what managers said they would do.
- Read the most recent 10-K, see what actually happened to each company.
- Categorize in what ways managers delivered or didn't deliver and estimating success rates.
Sometimes our results are pretty dramatic, because the managers are that bad:
"Spectacularly unreliable" is a fair assessment. Source: Stockfisher
What I Found
- Managers are systematically unreliable. (Surprise!) Their growth targets usually aren't met. They are bad forecasters.
- Managers are surprisingly good at following through on operational plans, cutting costs, divestitures, and most interestingly, M&A plans.
- We see huge variation in manager quality amongst S&P 500 companies. And we forecast companies with more reliable managers will significantly outperform those with less reliable managers, as defined by whether they follow through on what they say they will do.
- Company valuations can vary by 5-15% based on manager reliably alone.
Let me go into a bit more detail, and give some examples from our final forecasts where management went right and wrong.
As I wrote above, I found managers excel at operational execution. When the goal is controllable and internal, success rates are as high as 85-90%. Cost reduction programs almost always succeed, which surprised me. A few companies that showed this, where you can read their old 10-Ks and see the cost cutting work basically as stated: Ametek, Honeywell (despite having other huge failures), Parker Hannifin (which expanded margins even during flat growth). Emcor doubled margins versus their own conservative guidance. The pattern I noticed ws that when success depends on internal process improvement rather than external market dynamics, managers are pretty reliable.
M&A integration surprised me. Despite popular belief that acquisitions destroy value, I found 75-80% of teams successfully delivered promised synergies. A few examples, with links omitted but this can be pretty easily verified: S&P Global's IHS Markit deal exceeded targets by large margins. Fiserv delivered on-time. Charles Schwab fully realized $2B in TD Ameritrade synergies. Marsh McLennan integrated JLT exactly on schedule. I felt like, in hindsight, synergy targets are typically conservative, some sort of under-promise and over-deliver messaging strategy.
Another area where managers on average do very well is balance sheet management, with an 80-85% success rate, very roughly speaking. Leverage targets, credit ratings, and capital structure goals are tend to get met with high consistency. Capital return commitments are honored 75-80% of the time. Some examples: Coterra, ConocoPhillips, Nucor, Linde, all did great looking back at their 2020 - 2022 10-Ks. Financial discipline is just something managers are good at, on average.
Strategic divestitures work 70-75% of the time. I conclude that when management decides to exit businesses, execution is generally strong. Roper Technologies successfully transformed their portfolio, divesting a third of revenue. Johnson & Johnson's Kenvue spin-off went very well. Exelon's Constellation separation completed on schedule. So when I see managers claiming now that they will divest something, I tend to believe it.
I find that near-term guidance shows ~70-75% accuracy, defined as being within 5% for quarterly and annual targets. But sandbagging is pervasive. Companies like Danaher, Amphenol, and Brown & Brown routinely set conservative targets they beat by 30-50%. As wit M&A, management teams seemingly learned to set beatable annual numbers while missing long-term targets.
Now, for what I find managers are systematically wrong about.
Managers promising long-term revenue growth has a 30-40% success rate. This is the single most unreliable category. And, along with margins (see below, also not good), it's the most important one in our forecasts. So here, I suppose the conventional wisdom is right, that these aren't worth the paper they're written on.
I noticed a characteristic failure pattern in that things that, in hindsight, were clearly cyclical, managers extrapolated as if the growth would continue. Examples: Texas Instruments set 2027 targets at the 2022 cycle peak and was catastrophically wrong. Albemarle spent $7B in capex at the lithium cycle peak and took $1.1B in write-offs. Pool Corporation raised targets in March 2022 at peak and then revenue contracted 30%. Mohawk spent $650M on expansion at the 2021 peak and was forced to close new capacity.
Mean reversion, anyone? I could list endless examples but put more simply, most such things are bad, so it's more the exceptions where managers correctly predicted long-term revenue growth that would deserve a list!
I also think managers are bad at understanding market shares. They consistently overestimate how much of the future market they'll get. Intel's foundry strategy catastrophically failed and the CEO departed. Nike lost 3 points of market share during their failed direct-to-consumer strategy. Southwest was forced to abandon their 50-year strategy under activist pressure. Starbucks saw China share collapse from 34% to 14%. Incumbent market leaders consistently underestimate competitive threats and overestimate their strategic advantages. Which I guess is well known, but it's nice to see in the data.
I find that managers' margin expansion targets succeed less than 50% of the time. Managers seemingly always underestimate cost inflation. 3M couldn't forecast $16B in litigation liabilities. AT&T materially underestimated secular decline pressures. General Mills saw 66% earnings per share decline despite executing on strategic initiatives.
Strategic transformation has only a ~40% success rate in this 50 company sample. Digital and technology transformations particularly disappoint. Carrier's connected devices initiative achieved less than 6% of their 2026 target. NetApp's public cloud annual recurring revenue missed target by 35%. Seagate's storage-as-a-service generated negligible revenue and was abandoned. The strategies may be directionally correct but there is some planning falacy going on here.
Another obvious finding is that geographic expansion is significantly more difficult than management teams communicate. A.O. Smith's China expansion completely reversed into declines. Biogen had failed international launches. IQVIA's international segments dramatically underperformed. Marriott's China doubling strategy resulted in revenue per available room declines.
I think diversification plays almost always underperform too. Altria had repeated failures in smoke-free products with JUUL, IQOS, and NJOY. Goldman Sachs abandoned consumer banking after $4B investment. Stanley Black & Decker's failed portfolio plays led to 69% earnings collapse. Take-Two took a $3.5B impairment on the $12.7B Zynga acquisition. When core competency is in one business model, attempts to diversify almost always underperform guidance, and I don't think people should believe such plans by default.
Lastly, in 2025 this doesn't matter so much anymore, but ESG (or more generally environmental) targets fail most of the time. This doesn't matter much for forecasting company fundamentals though, so it's not so important to predict, at least not for us.