Betting on Unknown Unknowns
Betting on Unknown Unknowns
How to be unrealistic and predict the future
This year, I’ve had the great luxury of spending time with Jeff Wilke, former CEO of Amazon Consumer. One of his greatest strengths is his willingness to bet on the future and embrace optimism. Out of that meta-learning, I wrote the following post for the Scale team earlier this year—I hope you enjoy.
On the COVID Vaccine
I got the 2nd shot of my Pfizer vaccine in May, a little more than 1 year after the United States went into a full lockdown. On the one hand, it felt like an incredibly long time, but the vaccine has come faster than anybody could have realistically imagined. The speed of the COVID vaccine is one of the great miracles of modern technology.
In April 2020 in the height of the pandemic, New York Times reporter Stuart Thompson produced an interactive article to discuss the “realistic” timeline of the COVID vaccine. At that time, while Dr. Fauci among others were publicly touting a goal timeline of August 2021, his research had shown the “typical” timeline would be November 2033. He even created a number of switches on various things which could accelerate the timeline, but if you play with it, the fastest you can get the vaccine based on the simulation is 2023.
Now, what actually happened was even faster than the August 2021 timeline, and many of us received two shots in May 2021. So was Thompson just being ridiculously pessimistic? Did he not know what he was talking about at all? Not really—he produced his article after interviewing multiple vaccine experts, and he even shows the histogram of how long it took for other vaccines to be developed—it usually takes over a decade!
Based on all the data that Thompson presented, most reasonable readers would actually conclude that it was Fauci who was being crazy. Humanity has never created a vaccine in under a decade, and public officials are telling us that it’ll take a year?
But Fauci was right. In fact, he was slightly pessimistic (August 2021 vs April 2021). To have properly predicted the timeline in this case, you would’ve had to have bet on the fact that the mRNA vaccine technology was going to work, which in early 2020 was viewed as experimental. Moderna, a company built and founded in 2010 on the technology, didn’t have a successful drug until the COVID vaccine.
This case study on the COVID vaccine puts right at the forefront the tension between predicting the future and being realistic. These two things are not the same.
Predicting the Future
The future is difficult to predict. As the pace of technology has accelerated, the rate of surprises has as well. It’s not hard to find examples of experts’ predictions which always end up being way off. It’s not because the experts are not knowledgeable—in fact it might be because they are too knowledgeable.
There’s a peculiar thing where oftentimes it’s the wildly optimistic predictions that end up being right. Such predictions seem entirely crazy at the time—it feels like betting on everything going absolutely perfectly. On the contrary, they are betting on “unknown unknowns” which will meaningfully change the game.
The maker of these optimistic predictions will never be able to chart the path in which their prediction will come true, or even if they tried, that path will be wrong. But, the prediction will end up being right because they are properly betting on unknown unknowns. These future moments of human invention will surprise us each time, but it should not be surprising that they will happen.
On the other hand, making long-term predictions about the future which are grounded in reality will often lead you to be wrong. The world is not an automaton which continues deterministically based on current realities. Betting against invention and innovation has, on the whole, turned out to be a bad bet over the course of human history.
Accurately predicting the future relies on betting on unknown unknowns. It’s impossible to say what these breakthroughs will be, but it’s almost a certainty that they will happen.
How does this keep happening?
Perhaps one of the greatest examples in modern history of “betting on unknown unknowns” is Moore’s Law (the prediction that every two years, the number of transistors on microchips will double). This has been an unstoppable force, enabling a 50 million fold increase from 1965 until now.
This is a great example of betting on unknown unknowns. When Moore first formulated it in 1965, we did not know all of the innovations or feats of engineering that would have enabled this trend for the next 55 years. But, Moore was right, and every year we have continued to make advancements in research to continue to drive meaningful improvements.
The divergence between prediction and reality has continued, where in 2010 there were increasingly opinions from experts that Moore’s law was slowing or over. But, in line with the unstoppable power of unknown unknowns, we are continuing to keep pace into 2021.
Another shocking recent example was GPT-3 from OpenAI. For a long time, members of the OpenAI team were insistent that the long-term goal of AGI was going to be possible, and to many in the industry that seemed a little crazy. Then in 2020, they showed incredible capabilities with GPT-3, particularly in zero-shot and few-shot learning—capabilities which most researchers might have said we weren’t close to. To many, this was the first real glimpse at AGI.
These results would not have been possible had it not been for the Transformer model which came out of Google in 2017. This “unknown unknown” was difficult to predict the exact form of in 2015 when OpenAI first got started, but enabled a gain of two orders of magnitude on training efficiency and made the GPT-3 results possible. Betting on unknown unknowns worked out well.
The story of Amazon and AWS is probably one of the greatest stories of modern business, and if it were to happen in a TV show, the narrative would likely be criticized for jumping the shark and being unrealistic. What started as an online bookstore then became the largest internet marketplace in America, and then became the largest enterprise company in the world.
To have properly predicted the meteoric rise and growth of Amazon, you could have either been blindly optimistic, or you could have fallen in line with the pundits and been pessimistic on the stock. As a true unknown unknown, Amazon invented AWS which is now significantly more profitable than Amazon.com, and is responsible for nearly half of Amazon’s market cap.
At Scale, we’ve discovered unknown unknowns many times, and chances are that we will discover them again in the future. The discovery and deployment of technology to our annotation systems have allowed us to achieve significantly higher quality and lower costs, which have allowed us to continue to scale and compete across both dimensions. At the same time, we’ve built new products like Nucleus, Rapid, Document AI, and Mapping, each of which has allowed us to tackle new problems for our customers. I certainly did not predict that these innovations would come, nor that they would become critical to our continued ability to delight our customers, but here we are.
Memes, memes, memes
You could construct IQ bell curve memes around each of these, because the wildly optimistic prediction oftentimes seems “stupid”. People who are too pragmatic for their own good get too bogged down in reality, which leads them to be wrong.
For the development of AGI and the rolling trend of AI become ubiquitous:
For the unstoppable trend of Moore’s law:
And lastly, for Scale, infrastructure for AI:
Go, go, go, go
As a final caveat, you cannot always bet on unknown unknowns. Chances are that JC Penney is not going to magically become one of the most valuable stocks of the decade. If you had to bet on that one, the correct bet is probably that they will continue to be more and more irrelevant over time.
You can only bet on unknown unknowns near the frontiers of human tenacity and creativity. Towards the tail distributions on both traits, unexpected tail events start to happen that dominate everything else. The problem with JC Penney isn’t that the department store business is a bad business (you could argue that the online bookstore business for Amazon was also a bad business). The problem is that JC Penney doesn’t have a high density of people who are either deeply tenacious or deeply creative. The unknown unknowns at JC Penney are 1,000 times less likely than at Amazon, so you likely shouldn’t be betting on them.
When you have groups of people who are especially tenacious and especially creative, magic happens. Amazonians had to be far more tenacious than competitors, otherwise they would die. Amazon’s tenacity has perpetuated into Amazon being one perhaps the most innovative company in the world, continuing to invent and enter new markets at a breakneck pace. At its core, the reason to bet on Amazon is their corporate tenacity, and therefore inventiveness.
A bold vision can sometimes be confused with arrogance or hubris, and sure enough, sometimes that’s how it ends up looking. Betting on unknown unknowns still requires a deep ability to execute. But Scale has a track record of delivering unknown unknowns, and it would be the wrong bet to think that will stop.
Scale is trying to predict the future, so we bet on unknown unknowns.
Thank you to Jeff Wilke, Sam Altman, Pedro Franceschi, Victor Lazarte, and my Scale teammates for reading earlier drafts and providing feedback.