Messiness is Efficient
François Chollet | The NFL Hasn’t Been ‘Moneyballed’ Yet | Factoids | Elsewhere
Quote of the Moment
If you only have to deal with known quantities, strict organization is efficient. But if you have to deal with a lot of change and uncertainty, messiness is efficient.
| François Chollet, via twitter
The NFL Hasn’t Been ‘Moneyballed’ Yet
I read an deeply engrossing article by Alec Lewis of The Athletic — NFL teams know the best way to draft, so why aren’t they doing it? — that explains in considerable detail that NFL teams’ management is aware that they are approaching the NFL draft all wrong, but reject the analysis that could set them straight.
In a nutshell, economic analysis — with a large measure of behavioral economics mixed in — shows that NFL teams don’t value the options available to them in the draft in a rational way. They are overconfident in their ability to judge athletes, overvalue the earliest selections, fail to recognize that the draft is as much a lottery as a test of analytic techniques, and lack any comprehensive system to measure how good their decision-making is retrospectively.
Here’s Lewis:
[Steve] Gera left the NFL a decade ago and has since worked in the NBA and European soccer, founded data science companies and taught. Experience in different sectors helped crystallize some of Gera’s beliefs about football, and the Manziel moment epitomizes what Gera believes is one of the most faulty decision-making processes in the NFL: draft strategy.
Compare a prospect to a legend from the outset, and you — or, say, Browns owner Jimmy Haslam, who drafted Manziel No. 22 overall and then watched as the quarterback’s career imploded suddenly and spectacularly — are likely to cling to that early comparison despite evidence to the contrary.
“The draft is an absolute petri dish for every cognitive bias underneath the sun,” Gera said.
Conversations with 14 general managers, coaches, analytics staffers, scouts and executives in other sports — some of whom were granted anonymity because they were not authorized by their current organizations to speak about the highly competitive process — unearthed a messy concoction of uncertainty, overconfidence, competing incentives, pressure and impatience.
“Human dynamics writ large,” said Hall of Fame NFL executive Bill Polian.
Even Nobel Prize-winning scholars have spent decades mulling whether there is a single best way to draft.
The answer, they’ve found, is a resounding yes. But only a few teams are curious enough to think differently, and even fewer are disciplined enough to act differently.
What do those scholars have to say about the draft, and how should NFL teams change their approach? I will avoid the math, and distill what Cade Massey of Duke and Richard Thaler, an economics Nobel laureate at the University of Chicago, wrote in Overconfidence vs. Market Efficiency in the NFL Draft.
TL;DR: Unless NFL teams put into place sophisticated systems to counter human cognitive biases — especially those inherent to a set-up like the NFL draft, with dozens of teams contending for hundreds of athletes each of which has thousands of data points to assess — they will fail to accurately weigh the true costs of players relative to their likely success in the six years following the draft.
In particular, teams overvalue players selected in the early rounds and undervalue those in later rounds, and as a result teams ‘trade up’ too much to get those earlier draft players, and fail to ‘trade down’ to acquire less expensive and more undervalued players.
One feature of the systems they should put in place — but seldom do — is to create a regression on their bets made on players once they have played in the NFL, and use that league data to temper the valuation of picks in future drafts: in essence, to learn to make rational, not intuitive, decisions.
Long version (somewhat wonky):
The NFL draft involves a series of player selections, and — to state the obvious — when a team uses an early draft pick to select a player they are forecasting that the player will do well.
Teams often trade picks, for example giving up the 4th pick in exchange for the 10th and 21st picks, or in exchange for the 10th pick this year and two picks, 12th and 14th, in next year’s draft. These trades represent the market value of draft picks. Massey and Thaler define ‘surplus value’: ‘We define surplus value as the player’s performance value – estimated from the labor market for NFL veterans – less his compensation. In the example just mentioned, if the market for draft picks is rational then the surplus value of the player taken with the 4th pick should equal (on average) the combined surplus value of the players taken with picks 10 and 21.’
If the teams knew nothing about players’ future value, the draft would simply be a lottery, and early picks would be no more valuable than later picks.
But, as the authors reveal, ‘In the actual NFL, early picks are highly valued, despite little evidence that teams can effectively value player in the draft.’ The authors came to this conclusion after a deep analysis of early draft picks and years of their actual surplus value on the field.
They give the example of Ryan Leaf, who was taken in the pick following Peyton Manning, a future superstar, meaning Leaf’s surplus value was rated very high indeed. But Leaf flamed out: ‘after trials with several teams he eventually left the league a declared flop.’
The draft ‘market’ is flawed in key ways. There is no way to short sell the picks, and successful teams do not gain access to picking early, because losing teams get the early picks, through an attempt by the league to level talent across the league.
The researchers’ results showed that the cognitive biases they expected to be at work were even stronger than they expected, and the inefficiency of the market was worse than expected: ‘We expected to find that early picks were overpriced, and that the surplus values of picks would decline less steeply than the market values. Instead we have found that the surplus value of the picks during the first round actually increases throughout the round: the players selected with the final pick in the first round on average produces more surplus to his team than than the first pick, and costs one quarter the price!’
‘Very high draft picks are expensive in two ways: high picks can be traded for multiple lower picks, and high picks are paid higher salaries.’
‘We find that performance is correlated with draft order, as expected. Players taken in the first round are more likely to be successful (be on the roster, start games, make the all-star game) than players taken in later rounds. However, performance does not fall as steeply as the implicit price of draft picks.’ So, ‘late-first-round draft picks generate more value than early-first-round-picks.’
Those involved in draft picks rely too much on ‘intuitive predictions’ based on current evidence — like a student taking a ‘concentration quiz’ — and then predicting and end state — like that student’s grade point average after four years. Intuitive predictions are too extreme than is justified by the evidence. NFL teams do not apply ‘normative models’ — like the distribution of grade-point averages in the student example — so they tend to jump directly from evidence to prediction. They should ‘combine evidence about the player’s ability (his college statistics, scouting reports, fitness tests, etc.) with the prior probabilities of various levels of NFL performance to reach a forecast’, which they don’t generally undertake.
‘Overconfidence is a closely related concept in the psychological literature. Simply put, people believe their knowledge is more precise than it is in fact.’ This is true of scouts, GMs, and owners.
‘When people have more information on which to base their judgments their confidence can rationally be greater, but often information increases confidence more than it increases the actual ability to forecast the future.’ Having access to more information about players increases the ability to forecast, but overconfidence increases at a higher rate, outstripping their judgment.
The draft is in effect an auction, and ‘It is well known that in situations in which many bidders compete for an item with a common but uncertain value then the winner of the auction often overpays.’ ‘Rational bidders should recognize this adverse-selection problem and reduce their bids, especially as the number of other bidders increases.’ This is called the ‘Winner’s Curse’.
‘A related phenomenon occurs when a single party selects from multiple alternatives. If there is uncertainty about the true value of the alternatives, the decision-maker, on average, will be disappointed with the one she chooses.’ This ‘expectation inflation’ means ‘In the context of choosing football players, the implication is that the more players a team examines, the better will be the player they pick, but the more likely the team will be disappointed in the player.’
‘These problems are exacerbated when uncertainty increases and when the number of alternatives increase. The NFL draft is a textbook example of such a situation – teams select among hundreds of alternative players, there are typically many teams interested in any given player, and there is significant uncertainty about the future value of the player.’
False consensus is when someone believes others value choices to the same degree that they do. ‘False consensus suggests that teams will overestimate the extent to which other teams covet the same player, and therefore overestimate the importance of trading-up to acquire a particular player. Such a bias will increase the value placed on the right to choose.’
‘Considering that any one of the psychological factors discussed in this section could strongly bias NFL teams, we would not be surprised to find the NFL draft as “sloppy and inefficient” as [Michael] Lewis found major league baseball.’
Michael Lewis wrote in Moneyball:
If professional baseball players, whose achievements are endlessly watched, discussed and analyzed by tens of millions of people, can be radically mis-valued, who can’t be? If such a putatively meritocratic culture as professional baseball can be so sloppy and inefficient, what can’t be?
Well, no surprise, the NFL is as sloppy and inefficient as baseball. We can expect more Ryan Leafs. And Tom Brady was the 199th pick in the sixth round of the 2000 NFL Draft, which shows how poorly the NFL Draft can value prospects.
The Larger Business Implications?
The authors expend a paragraph on some telling observations about the wiser applicability of these findings:
The implications of this study extend beyond the gridiron. Football players are surely not the only employees whose future performance is difficult to predict. In fact, football teams almost certainly are in a better position to predict performance than most employers choosing workers. Teams get to watch their job candidates perform a very similar task at the college level and then get to administer additional tests on highly diagnostic traits such as strength and speed. Finally, once hired, performance can and is graded, with every action visible on film from multiple angles! Compare that to a company looking to hire a new CEO (or an investment bank hiring an analyst, a law firm hiring an associate, etc.). Candidates from outside the firm will have been performing much of their job out of view. Outside observers see only a portion of the choices made, and options not taken are rarely visible externally. And, even once a CEO is hired, the company’s board of directors is unlikely to be able [to] measure his or her performance nearly as accurately as a team can evaluate its quarterback. In our judgment, there is little reason to think that the market for CEOs is more efficient than the market for football players. Perhaps innovative boards of directors should start looking for the next Tom Brady as CEO rather than Eli Manning.
On the other hand, many of the stresses built into the NFL Draft are relaxed in an everyday work context for judging candidates, and the contrived elements of the Draft — such as more successful organizations being blocked from competing for the best candidates — don’t apply. But companies don’t seem to move past intuitive forms of candidate evaluation and seldom track how well their predictions of future performance turn out, years later. Some of that is due to the short-sightedness of management, but some is due to the fairly chaotic world of business, where people are constantly coming and going, companies merge, divisions are shut down, and reorganizations unmake decisions earlier made by now absent CEOs.
A wicked problem, and therefore one that can’t be ‘solved’ but only ‘tamed’, as John Camillus pointed out, ‘Increasingly, these are the problems strategists face—and for which they are ill equipped’. But to the degree possible, organizations would benefit from the advice and wisdom of Massey and Thaler.
Factoids
Until the year 2000, about 25% of top-grossing movies were prequels, sequels, spinoffs, remakes, reboots, or cinematic universe expansions. Since 2010, it’s been over 50% ever year. In recent years, it’s been close to 100%.
| Adam Mastroianni, Pop Culture Has Become an Oligopoly
The blanding of everything.
…
About three quarters of the 10.5 million undocumented people in the United States as of 2021 were in the labor force. Roughly two million people out of the overall undocumented population have a temporary legal status that makes them eligible to work.
…
In Jefferson’s time, most Americans were farmers. By 2019… only 7 percent of rural Americans were.
Partners and Sponsors
I attended Social Now a few years ago and I'm glad to see it's back. And it's the 10th edition already!
Want to optimise your digital platforms and your organisation's digital experience to improve internal communication, collaboration and organisational learning? Get yourselves to "Social Now 2024: Realising the value of your digital workplace". 16 & 17 May, in Lisbon, Portugal.
…
I saw that Optionality’s founders — Elisa Camahort Page and Jory Des Jardins — interviewed my old and dear friend Cathy Brooks, who explored the (hilarious) topic: ‘How to find your voice and use your voice without being an asshole.’
Elsewhere
Unions
Robert Reich lays out the thesis that the labor movement ‘pendulum is now starting to swing back’ after the decline that started in Reagan’s administration.
I believe the pendulum will continue to swing toward unions. Which means better wages and working conditions, a larger middle class, and laws and regulations that benefit the many rather than the few. And the possibility that America’s working class will return to the fold of the Democratic Party, where it belongs.
At a time when there’s a lot to be disillusioned about, this is the clearest, most positive trend in America.
I’m with him. I am working on a long, timeline-centered piece (or series) on the history of unions — specifically auto unions — in the U.S. Stay tuned.
Private Equity
Eric Gardner zooms in on private equity, which now dwarfs public markets in funding companies:
For most of the 20th century, the easiest way to raise a large amount of money for a business to fund expansion and operations was through a public stock offering. In exchange for the ability to sell their stock to the public, management has to regularly provide financial information, which is analyzed and scrutinized by regulators and investors.
But over the past three decades, private equity has overtaken the public markets. According to JP Morgan’s research, there were 7,300 public companies in 1996; today, there are just 4,300, a decline of over 40 percent in less than 30 years. During the same period, the number of private companies backed by private equity firms rose almost 500 percent to 11,200.
Unlike public companies, the finances of private equity-backed corporations aren’t subject to regulatory or disclosure requirements. “With fewer companies listed, there may be a decrease in overall transparency and investor trust in the market,” an investment banker told CNN.
[…]
Last August, the Securities and Exchange Commission passed new reporting requirements for private equity and hedge funds, which typically seek big returns through high-risk investment strategies on behalf of wealthy individuals. In a 3-2 vote over the objection of the commission’s two Republican members, the SEC adopted new reporting requirements and changed how the industry interacts with its customers. It’s estimated that over $25 trillion of pension plans, endowments, and individual assets are managed by these funds.
Yes, $25 trillion. That’s a lot of zeroes.