Serendipity As A Science
Surfaces For Opportunity
In computer science — or engineering in general —there’s a lot that can go wrong. From vulnerabilities in cybersecurity to synchronization bugs in app design, systems have become so complex that no amount of thought or math can ever prove them free of error, and designers have been forced to retrench. By recognizing that failure can come from anywhere, they work to reduce the exposure of individual sub-systems, accepting that some components must be allowed to crash without bringing the whole system down. These areas of exposure, conceptualized as “surfaces”, are to be made as small as possible. For cybersecurity, this may mean reducing the number of accessible ports, not because they have any reason to believe an individual port is insecure, but because more access points provides a higher attack surface. For an app designer, it may require frequent backups to avoid potential data loss, while for a tower architect it may include adding redundant structures to avoid single-point failure. In all these instances, designers avoid exposure to unpredictable events by minimizing their impact area, and because they do, we can build a World Wide Web on buggy servers and (usually) expect it to work.
It’s tempting to assume that reducing exposure to unpredictable events is good for life in general. After all, by purchasing insurance, we reduce exposure to sudden financial damage, and by getting an annual checkup, we reduce the odds that we’ll die of cancer. However, protection is only one side of the coin. Many of life’s greatest opportunities are products of serendipity, arising unpredictably and accessible only to this in the right place at the right time. Think of the job offer that nearly doubled your salary, or the chance encounter where you met your partner. Neither could have been predicted in advance, and worked out in your favor because they happened when you were ready to take advantage of them. Positive and negative serendipitous events are highly symmetric, and the dangerous failure surface implies a positive dual, the “opportunity surface”. In the rest of this post, I will provide the conceptual framework for such opportunity surfaces, define their connection to antifragility, and finally describe some ways to maximize them and expose ourselves to life-changing opportunity. Let’s dive in.
Antifragility
Think of the last time you worked out. Whether it was hours or months ago, you probably experienced physical strain, compressing your joints, introducing micro-tears into your muscles, and (if you’re like me) flooding your mind with shades of “Help me, God” and demanding rolling outlays of willpower. Yet you did not die. Unlike the teacup which shatters when subjected to stress (read “looked at wrong”), your tendons thickened, your micro-tears healed stronger, and your capacity for willpower actually increased. When subjected to a non-lethal stress, you didn’t crack, but actually came out stronger. You are antifragile.
Nassim Taleb, the guardian angel of this blog, was the first to directly define antifragility. An object is antifragile if it benefits from increased volatility in its environment, while it is fragile if it suffers. The clunkiness of the name reflects the novelty of the idea, as the English language assumes volatility is pretty much always dangerous, with objects ranging from perfectly fragile to perfectly robust based on their susceptibility to it. We take it for granted that certain physical or mental stresses will cause us to adapt and grow stronger, but we rarely compare that response to those of other objects, most of which become damaged and break. Why the difference? Antifragility arises primarily in complex systems, particularly those with the capacity for adaptation. Look at the evolution of species. A mutation may on balance result in the death of an animal nine out of ten times. However, if that one time results in an animal better suited to its environment (or at least the tastes of its mates), then that mutation will spread through the species and cause it to change. Under environmental pressure, once lethal mutations may suddenly become advantages (think fur in a hot climate which suddenly gets cold), allowing the species to continue. In this way, evolution follows a sort of gradient search, improving the species in response to change. Rapid environmental changes — increased volatility — result in more generalized adaptations, as those fitted for one change may be lethal in the next, resulting not just in faster evolution, but generally better evolution for those species which survive. Think of the rapid environmental changes our ancestors experienced as they explored the globe, prompting the development of language, technology, and eventually modern civilization. Through such natural selection, life is antifragile.
The body is similar — damage results in adaptation, with individual cells sacrificing themselves for the good of the whole. Society, too, benefits from shocks — think of how many improvements to the human condition came about as the results of revolution (American, French, even Russian if you’re feeling like a fight). A more technical example is simulated annealing, a computer pathfinding technique in which an algorithm finds a local maximum, is “shaken” out of it by a random, large movement, then continues wandering along its gradient, ideally to a higher maximum. The sudden shocks and increased volatility improve the result. A teacup, on the other hand, has no capacity for adaptation, and can only suffer from stress. Even if it doesn’t crack, every blow results in microscopic structural damage, adding up to ultimate failure.
So what does this have to do with opportunity surfaces? Generally, a system is not simply fragile or antifragile. Rather, there are many dimensions corresponding to different types of volatility — different surfaces. A system which is fragile to shocks along one axis treats all exposure to those shocks as a failure surface, the area of which it aims to minimize. Symmetrically, along an axis with antifragile characteristics, volatility is generally beneficial and we aim to expand the corresponding surface to capture as much opportunity as possible.
Staying Ahead of The Curve
Now that we know the difference between fragile and antifragile systems, how do we identify them? Furthermore, how do we determine which types of exposure are worth maximizing? The answer lies in convexity.
Mathematically, a function is convex if the line between any two points lies “above” the curve. If the opposite holds true, it is concave. Intuitively, lines which increase in slope as the underlying parameter increases (i.e. as we move to the right) are convex, while those which decrease and level off are concave. These abstractions translate easily into the real world. Think about the relation between the cost of wine and its taste. Those of us who rarely drink can still tell the difference between $5 bathtub hooch and a $40 Napa Merlot, but the difference between that Merlot and its $1000 French equivalent is far more subtle. Despite the massive price increase (movement to the right), there is a lot less quality improvement (movement up). Thus, wines are concave with respect to price, or — equivalently — follow the Law of Diminishing Returns.
Most systems are fairly concave, at least at the beginning, with the first motion to the right resulting in the biggest gains, then leveling off afterwards. However, many experience a period of convexity a little ways afterward. Look at education. It the beginning, the gains from learning such simple concepts as reading and basic math far outweigh the gains from the later laborious efforts of learning correct spelling, grammar, and trigonometry. This early convexity is the source of much frustration, with students grumbling about topics they’ll “never use outside of school”. This stretch of concavity continues through most of primary and secondary education, with only an artificial kink at high school graduation, since many non-college jobs require a diploma. After this, however, it begins to slope up again, as differentiation takes hold. In technical school or college, students learn skills which place them above their peers in a (hopefully) valuable way, and increased effort, leading to increased differentiation, can now suddenly catapult them to massively outsized returns. Imagine the computer science student who sticks with his cohort, putting in 40 hours of effort per week, and graduates with enough knowledge to earn the average starting salary ($66,631 per annum at the time of this writing). Now imagine his peer, who puts in 50 hours per week, compounded over the length of his degree, and is now firmly in the top 20% of applicants. Instead of competing for offers, he has room to negotiate, and can pull maybe $90k per year. Finally, the workaholic who puts in 60 hours per week lands squarely in the top 5%, gets a phenomenal offer from a FAANG company, and pulls $150k plus equity right out of school. Because starting salaries follow a convex distribution, the extra 20% increase in effort brought a 50% return. I get it now, Dad.
So barring a return to school, how can we find and exploit convex systems? In a symmetry that would give Andrei Sakharov nightmares, concave systems vastly outnumber their convex counterparts, and even when convexity appears, its range is usually limited (think of the increasing utility of a weight-lifter piling on weight until the sudden plummet when it becomes more than his body can bear). How can we find these rare opportunities, especially those with a long runway of convexity?
It turns out that there are two predictable types of convexity, each with its own giveaways, and each based primarily on a cognitive bias. The first arises from compounding, in which subtle positive feedback loops can be harnessed to generate outsized returns. The second is based on quirks in the definition of expected value, and shows up wherever a few people get extremely lucky. Let’s start with compounding.
Gradually, Then Suddenly
Humans think linearly. In most day-to-day situations, there’s a strong direct correlation between the size of an input and that of the output. If I run twice as long, I’ll cover twice as much distance. If I increase my salary by 20%, I can afford a moderately nicer car. If I eat 10% less ice cream, I’ll stop slowly gaining weight. With linear phenomena, the only way to get large outputs is through large inputs, whether in time, money, or effort. You want to be a millionaire? Gotta work a lot harder. As if that doesn’t already sound hopeless, the Law of Diminishing Returns lurks right around the corner, eating away at extra gains like a sad reality tax.
Nonetheless, the world is full of broke people and billionaires, and no matter what certain political apologists will tell you, one did not work billions of times harder than the other. Even correcting for advantages of family, talent, or luck (which we’ll get to shortly), these discrepancies are too vast for a linear explanation. Enter exponentiality.
An exponential system is one in which change is proportional to current value. Check out this graph:
Humans are great with linear functions (50x) or even polynomial ones (x³). However, we are terrible with exponents. Look at how small the 2^x curve remains for the first half of the graph — if this were a comparison of investment strategies, it would appear to be the clear loser. However, something interesting happens past about 6: the exponential curve explodes. In the span of just a couple digits, it rockets from the laggard to absolute dominance, with no sign of slowing down. Some simple math shows that even the slowest growing exponential curve will beat any linear or polynomial curve, given enough time. This idea is both obvious when graphed and notoriously difficult to grasp when we see it in real life.
“How did you go bankrupt?” “Gradually, then suddenly.” Hemingway’s lines apply not only to losing money, but surprisingly, to making it. Compounding is the art of using something to proportionally augment itself. The obvious example is finance, where most instruments are judged by their annual percentage return. However, compounding effects lie hidden all around us. Think of the social media post which gets a few shares, then a few more, then is suddenly exploding into virality as the share count compounds on itself. Or the speed of information acquisition as one learns more and more, with simple ideas providing easy construction platforms on which to build new ones, so complex concepts appear suddenly simple and clear.
All compounding phenomena are based on an effect’s increase in proportion to its current size, and follow an exponential curve. They initially appear to be “losers”, increasing only slightly despite large outlays of effort. However, past some inflection point, they suddenly take off, dwarfing other strategies in their ascent to the moon. Hence these phenomena benefit from any event, intentional or otherwise, which may push them into takeoff territory. They are extremely convex, and extremely antifragile.
To harness exponentiality, it is important to identify compounding phenomena, maximize our exposure to them, and then — above all — to be patient. Investing in stable, cash-paying equities may seem silly when compared to the outsized gains associated with hype stocks, but remember that given enough time, the linear increases associated with hype will be dwarfed by that exponential curve, provided the compounding effect remains stable. Or, when networking, remember that even if the first few friends you make may not be founts of opportunity, but their connections might, and the size of your circle will compound as word of your wisdom regarding opportunity surfaces spreads. When learning a new skill, the first hundred hours may feel slow and grueling, but during that time you’re forming the basis for takeoff, and the only way to get there is patience.
So now we’ve seen the power of compounding, but is that really all there is? If we want to experience outsized gains, are we cursed to buy 4% dividend stocks and wait until retirement to enjoy the results? Fortunately, no. Opportunity exposure is not limited to predictable gains. In the last section, we’ll explore phenomena which look a lot like exponential curves, but where the source isn’t compounding, but pure, raw luck.
Maximizing Serendipity
The second type of convexity occurs due not to a phenomenon’s self-acceleration, but rather its apparent unpredictability. These events generally massive expected outcome, but — due to their rareness or our inexperience with them — tend to appear not worth the effort. Let’s return to networking. Imagine that out of every ten people you shake hands with, maybe one will become a long term contact. For each five long term contacts, one will offer you an interesting opportunity. And for every five interesting opportunities, one will pay off big. For each person you go out of your way to meet, the odds of a big payoff are only 1/250. If you only meet a handful of new people every month, the odds of a big payout will remain small, and you’ll find yourself asking what the point is behind all these meetups. It isn’t until the 125th connection that your odds of a big payout (like a new job or investment opportunity) even cross 1/2. In this situation, your heuristic brain is going to have trouble reconciling the effort with a benefit that never seems to arrive, so unless you’re already a social person, you may quit long before you get results.
Mathematically, most of these opportunities arise from approximations of Zipf’s law, a probability distribution which resembles a “backwards” exponential. Think of the frequency of words in a language. The Second Edition of the Oxford English Dictionary contains entries for approximately 170,000 words in current use (and another 40,000 obsolete ones), meaning the expected frequency of a word selected randomly from the dictionary is roughly 1/170,000. The “from the dictionary” part is important, because the majority of these words don’t occur nearly that often in real life. Instead, a handful of incredibly common words, like “the”, “or”, and “maybe” show up orders of magnitude more frequently that “soliloquy” or “polydactyl”. Because of this crowding, the frequency of our dictionary word may be closer to 1/10,000,000 or even lower. The presence of a few extremely common words skews the entire distribution. Importantly, because most words exist in the long, low tail, selecting a couple dozen from a dictionary at random and taking the average of their frequencies is still very unlikely to give us a good idea of the expected frequency, since the odds of including one of the few extremely common words in our sample is very low. We may conclude prematurely that English is an odd language with millions of words, none of which are very common.
Now let’s apply that logic to real-world opportunities. Imagine that you have a lottery ticket which has a 1/1000 chance of giving you a million dollars, a 1/100 chance of giving you 10,000 dollars, and the rest of the time, gives you nothing. Each ticket costs a dollar. If you’re aware of these probabilities, you can determine that the expected value of the award is $1100, coming at a cost of only $1, and you’ll gladly play. Though most tickets will be worthless, the few that aren’t are so valuable that they skew the entire distribution.
However, what happens if you don’t get to know these probabilities? What if, instead of lottery tickets, they’re job applications? You know that a handful of people with your desired job title own big lake houses, but the first dozen applications you send for that position get rejected offhand. Would you give up? It’s very easy to do so — after all, the observed ROI is zero. What about after the first 30 applications? It’s possible that your odds of success are only 1/100, but that the return for such success would be a rough payoff of $5,000,000 over the next few years. The problems here are a) it’s difficult to judge your probability of success, and b) the hindbrain doesn’t like statistics, it likes narratives. Without the math to guide us, all we perceive is failure after failure, and our brains very quickly construct narratives around why we seemingly can’t get the position, and should cease trying. It’s incredibly easy to fall into this trap and just give up, as the penalty for doing so is invisible. We just go back to our old lives, unaware of just how close we were to victory.
I’m not recommending you spend the rest of the year submitting Hail-Mary job applications. Instead, I want you to walk away with the following lemma. Despite the low odds of success on any given Zipf Law-type endeavor, the expected return is very large, so any exposure to the phenomenon, no matter how small, is worthwhile. It may not make sense to submit a thousand job applications, but make sure to submit a few, now and again. Once you’ve identified a phenomenon with a Zipf distribution, find a method of expanding at least a small opportunity surface to meet it. It may never pay off, but if it does, you’ll be very glad you did it.
So how do we identify phenomena with a Zipf distribution? Find areas with a widely varied success rate that seems mostly due to luck. For instance, the average Tweet will get a couple likes from friends and family, but some tiny percentage will go viral, seemingly through nothing but positive feedback and network effects. Similarly, the world is full of small businesses, but a handful will grow to be Google. Most novel writers, CEO’s, and 1840’s gold prospectors share the characteristic that almost all of their success is due to luck, and the expected return of the endeavor is wildly larger than the overwhelming majority of participants experience. In a way, it is the inverse of the compounding phenomenon, in which a subtle advantage translates to an outsized return. Instead, you’re making it easier to get lucky.
Once we’ve identified a Zipf distribution phenomenon, how can we take advantage of it? As any exposure has a large rate of return, we should seek to maximize exposure while minimizing regret — i.e., putting in as much effort as we’re willing to, given the possibility that our net payout may be zero. In other words, if you have some spare time, write a blog post, learn a bit of a valuable skill, or go for a walk just thinking about new ideas. These activities are short and easy, yet provide a little surface for massive opportunities to arrive. Also, remember to have patience. Day after day, year after year, our heuristic hindbrains get fed up with even these small efforts due to the apparent lack of payoff. Stay strong. Through maximizing exposure to activities with high expected outcome, odds are high that someday, serendipity will deliver.
