Categories
reading

The Black Swan by Nassim Nicholas Taleb

The Black Swan by Nassim Nicholas Taleb
Nassim Taleb’s The Black Swan is a fantastic, eye-opening book that will challenge not only what you know but also what you think you know. Taleb is widely renowned as the guy who made beaucoup amounts of money off of the 1987 stock market crash. He profited not by predicting the crash would happen, but that the system would eventually produce a “black swan” event that would make options insanely profitable.

Recommendation — Nassim Nicholas Taleb (or “NNT” as I like to refer to him) opened my eyes with this book. Any book that can blow open your understanding of the world is a must-read — and this is one of those books. The role that randomness and unpredictability play in our lives is completely under-appreciated, when it is acknowledged at all. Just one attempt at appreciation I’ve made can be found in my post “But For,” which is an attempt to string together a series of unplanned events that have cumulatively had an enormous impact on my life.

Going forward, I want to garner a greater appreciation for power law, stochasticity, black swan events, and living in “Extremistan.” On my immediate reading list are related books: The Drunkard’s Walk: How Randomness Rules Our Lives, Stumbling on Happiness and The Luck Factor: The Four Essential Principles. I’ve yet to order it, but NNT’s first book, Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets, is also on my list.

I’ve become a bit of a Nassim Taleb junkie and typically link down stuff he puts out (videos, articles, posts to his low-tech blog “Opacity,” etc.).

Review — Rather than recant what others have said better, I’ll selectively quote a thorough and informative review of the book from Amazon:

The Black Swan is probably the strongest statement of enlightened empiricism since Ernst Mach refused to acknowledge the existence of the atom. Of course, in theory, everyone today is supposed to be an empiricist – all right-thinking intellectuals claim to base their views solely on positive scientific observation. But very few sincerely confront the implications of rigorous empiricism. Specifically, few confront “the problem of induction,” illustrated here by the story of the black swan.

Briefly: observing an event once does not predict it will occur again in the future. This remains true regardless of the number of observations one adds to the pile. Or, as Taleb, recapitulating David Hume, has it: the observation of even a million white swans does not justify the statement “all swans are white.” There is no way to know that somewhere out there a black swan is not hiding, disproving the rule and nullifying our “knowledge” of swans. The problem of induction tells us that we cannot really learn from our experiences. It makes knowledge very problematic, if not impossible. And yet, humans do behave -almost without exception- as though they believe that experience teaches us lessons. This is forgivable; there is no better path to knowledge. But before proceeding, one must account for the limits that the problem of induction places on our claims to knowledge. And humans seem, at every turn, to lack this critical self-awareness.

Taleb explains that conventional social scientists use induction to collect data, which is then plotted on the good old Gaussian bellcurve. With characteristic silliness, Taleb dubs the land of the bellcurve “Mediocristan” – and informs us that it is the natural habitat of the white swan. He contrasts Mediocristan with “Extremistan” – where chaos reigns, the wholly unexpected happens, power laws and fractal geometry apply and the bellcurve does not. Taleb’s fictional/metaphorical ‘stans’ share something with the ‘stans’ of the real world: very ill-defined borders. Indeed, one can never tell whether one is in the relatively safe territory of Mediocristan or if one has wandered into the lawless tribal regions of Extremistan. The bellcurve can only help you in Mediocristan, but you have no way of knowing whether you have strayed into Extremistan – beyond the bellcurve’s jurisdiction. This means that bellcurves are of no reliable use, anywhere. The full implications of this take a while to sink in, and are sure to cause huge controversy. In July, Taleb will debate Charles Murray (author of -what else?- the Bell Curve). I’ll let you know who wins.

Categories
linked down

Blogging Thesis Progress (On power law distribution)

http://polibyte.com/blog/blogging_thesis_progress

I’m pretty sure I already knew that hyperlinks and web traffic followed a power law distribution, but Brian’s explanation is clear and worth saving down for future reference.

Another possibility is that the Internet might not be so egalitarian after all. To understand why this would be, it’s necessary to reflect on the structure of the web. The element tying one web page to another is the hyperlink. Clicking a hyperlink is what allows an Internet user to “browse” from one web page to another. Across the web, hyperlinks follow a power law distribution . A power law distribution is highly inegalitarian; this means that a small number of web sites are the destination of the vast majority of hyperlinks.

The distribution of traffic to web sites also follows a power law. To understand why this should related to the hyperlink structure, it’s necessary to think about the ways Internet users discover web sites. If a user already knows about a web site, they can visit it directly. If they don’t, they can discover it via a hyperlink from a site they already know about or by using a search engine like Google. Both of these methods favor the discovery of highly linked-to sites. When browsing the web, the more hyperlinks there are to a site the more likely a user is to come across one of them. When using a search engine, most users only visit web sites on the first page of results. The release of search data for over 600,000 AOL users showed that 90% of clicks went to the results from the first page, 74% of clicks went to the first 5 results, and 42% of clicks went to the first result. This is significant because search engines’ rating algorithms give heavy weight to the number ofhyperlinks a site receives. Although the exact algorithms vary from search engine to search engine and are often secret, search engine result ordering is barely distinguishable from simply ordering web sites based on the number of hyperlinks to them.

Categories
articles

The Stochasticity of Life

A few months ago, various circumstances led me to a brief bout with depression. It is a strange moment when you reflect that, “I am depressed”. As I’m prone to do when I don’t know much about a subject, I googled depression. As part of my searching, I ended up reading some insightful comments on depression by Art De Vany, who almost three years ago today, was watching his wife succumb to a terminal disease. Regarding depression, Art wrote (emphasis mine):

Are you depressed they ask? And I say no, you only get depressed when you compare the present state with one that is better or perfect in some way. If you accept the reality of the present state, then you can’t make these irrelevant comparisons of what is against the ideal.

You are so strong they say. And I say, no. I am just grounded in the reality of the now and trying to find the best things to do to influence the ensemble of paths on which our lives will evolve from here. If I become depressed or confused, I give up our moment of power. …

The lifepath ensemble formulation is a liberating idea because it makes you understand that you cannot achieve a unique outcome and that the transitions from this state to the next are stochastic. All we can do is to do those things that make favorable transitions more likely.

Not that depression is that bad a thing always. If it is motivating to realize you have fallen short of some attainable goal, it may lead you to improve your preparation for the next life transition. But, if you think you can achieve the change or goal with certainty, then you may become depressed in an unhealthful way. This can fall into a non-linear dynamic that is reinforcing, leading to deeper depression and, eventually, non-competent decisions.

After reading these comments a few months back, I sensed their truth while rejecting their application. Depression is sinister in that it is addictive — I wanted to wallow in my depression rather than work to escape it. It is so easy to be the victim.

Though I won’t elaborate on this further here, it’s likely that what helped me overcome my depression were the positive steps I took1 to improve my health. Not surprisingly, these steps were small to start but have cascaded, compounding their goodness in a non-linear fashion.

Underlying the depression application of De Vany’s comments is a central, important idea: the stochasticity2 of life. De Vany also calls it the “lifepath ensemble formulation”3. These descriptors evoke imagery of a symphony of circumstances, many of which are unpredictable, that drive life forward. So much of life occurs as the sum of a randomness. Even when not random, the number of causal factors in life are often so great as to destroy predictability.

Rather than despair over life’s innate uncertainty and randomness, I accept it. Doing so assuages my anxiety about potentially negative outcomes and empowers me to take the necessarily small steps that will further progress towards a goal. This is even more important when the lifepath ensemble seems nothing more than cacophony. Indeed, the stochasticity of life adds depth and beauty. Would you really have it any other way?

Though I can’t be sure, I bet that is what De Vany means by the “lifepath ensemble” — it is the string of individual actions that come together to set a course for my life. The course will be anything but certain, so accept the uncertainty, and work within it.

Extracurricular reading

  • Anyone else reminded of Knocked Up? There’s a line in that movie spoken by the father (played by Harold Ramis) to Ben (Seth Rogen) that goes something like “Life doesn’t care about your plans!” Here’s the mp3 of Ben more or less reciting this line to Allison (Katherine Hiegl). A strong undercurrent of this hilarious movie is the stochasticity of life.
  • I might have to blog on this in more detail later, but Scott Adams, the creator of Dilbert, practices “affirmations”. Affirmations is a method whereby you write out a specific goal 15 times a day for as long as it takes (at least six months) for the goal to manifest itself in your life. Mystical enough for you? Adams says he practiced affirmations regarding becoming a syndicated cartoonist. His affirmation was “I Scott Adams will become a syndicated cartoonist.” Correlation is not causation; however, I can imagine a causal pathway whereby writing down a specific goal encourages you to focus on means to accomplish the goal, consciously or otherwise. Alternatively, focusing on the goal raises your awareness and helps you tune out much of the stochastic noise of life and focus on taking those small steps. Here is a re-post4 of Adams’ post. Note: I have not tried affirmations.
  • My sister clued me in to Earl Nightingale’s “Strangest Secret in the World”. Here’s a ten minute clip of this on youtube. You can listen to it but the secret is that “we become what we think about”. Again, the method by which thought becomes action is unclear; however, it seems obvious that we will actualize our desires and our desires spring from our thought. So think!

Footnotes

1 Art played an indirect, but prominent role in that process, as well, so I probably owe Dr. De Vany a “thank you” or two.

2 To save anyone from looking this up — as I had to, “stochasticity” means:

the quality of lacking any predictable order or plan

3 Such a provocative descriptor and yet he has not since blogged on the “lifepath ensemble” again — 1 2

4 Quite bizarrely, the original blog post by Adams on his Dilbert Blog has disappeared. You can do the googling yourself if you don’t believe me. Strikes me as odd. Note: in the original post, Adams alluded to a book on luck by Richard Wiseman. That post is still out there for the reading.