The Raving Theist

Dedicated to Jesus Christ, Now and Forever

Fooled By Randomness?

September 26, 2006 | 29 Comments

Option trader Nassim Nicholas Taleb bills his book Fooled by Randomness as being about “randomness disguised and perceived as non-randomness (that is, determinism).” His main theme is that people over-rely on the principle of induction and get burned when something doesn’t go according to formula. He introduces the subject as follows:

There is a problem in inference well known as the problem of induction. It is a problem that has been haunting science for a long time, but hard science has not been as harmed by it as the social sciences, particularly economics, even more the branch of financial economics. Why? Because the randomness content compounds its effects. Nowhere is the problem of induction more relevant than in the world of trading — and nowhere has it been as ignored!

Taleb places special emphasis on the “black swan problem” — the notion that no number of observations of white swans could disprove the existence of a black swan. Applying this notion to the financial context, Taleb notes that it would be an error to conclude that the market could never go down more than 20% in a three month period merely because it never has.

Taleb’s discussion confuses a number of basic concepts. First, induction is not a problem of science — in science, it’s simply the underlying assumption that the same causes will always produce the same effects. Whether there is any way of proving this assumption that the future will resemble the past is certainly a problem, but it’s one of philosophy rather than science. I think Hume had it right when he concluded that it is question that cannot be resolved through formal logic, but science isn’t concerned or “haunted” with whether Hume’s conclusion is true.

Second, it makes no sense to suggest that hard science could be “harmed” by the alleged problem. The harm would be in doubting the validity of induction at all, in expecting different effects from the same causes. In mixing hydrogen and oxygen to produce water, scientists aren’t concerned that one day a “black swan” result (say, the creation of gold) will arise. When there is a deviation from the expected outcome, it’s not attributed to a “random” suspension of the law but to a difference in the original or intervening conditions. And even with respect to black swans themselves, no one believes that they are somehow a violation of otherwise immutable genetic or biological laws.

Finally, Taleb’s conclusion that induction is less reliable in financial markets because of increased “randomness” misses the mark. As a threshold matter, in the social sciences induction is applied to a significantly different category of events than that involved in the hard sciences. The predictions do not concern the behavior of atoms and molecules and other forms of matter. Financial analysts don’t chart the position of every particle in each potential market decision-maker’s brain and employ principles of neuroscience and chemistry to forecast buy/sell orders and the resulting stock prices. Rather, they deal more broadly with the conscious reasons for buying and selling, which reasons may take into consideration the various reasons consumers may or may not buy the company’s products, as well as the perceived reasoning of the company’s management, employees, suppliers, distributors and competitors.

The reasons may vary wildly and in their totality present an impossibly complex scenario, but that there are impediments to predictability does not mean that there is randomness. Every contributing decision may be perfectly intentional and reasonable — one person might sell a stock for pressing tax reasons and another might buy based upon inside information. People may indeed overestimate their ability to account for all the decisions and calculate their effect, but this does not mean that they are mistaking randomness for determinism, or that they are confronted by any randomness at all.

Comments

29 Responses to “Fooled By Randomness?”

  1. "Q" the Enchanter
    September 26th, 2006 @ 11:03 am

    Agree that the “problem of induction” is a philosophical rather than scientific problem. Not sure I follow the rest of the argument. The fact that individual trading decisions are explicable in terms of reasons says nothing about whether the overall picture is random. For instance, we presume that ensembles of atomic events are random even though the individual atomic careers are explicable in terms of certain forces and laws. (Indeed, we consider the individual atomic careers themselves to be random, and despite their in principle explicability.)

    Am I missing your point?

  2. "Q" the Enchanter
    September 26th, 2006 @ 11:03 am

    Agree that the “problem of induction” is a philosophical rather than scientific problem. Not sure I follow the rest of the argument. The fact that individual trading decisions are explicable in terms of reasons says nothing about whether the overall picture is random. For instance, we presume that ensembles of atomic events are random even though the individual atomic careers are explicable in terms of certain forces and laws. (Indeed, we consider the individual atomic careers themselves to be random, and despite their in principle explicability.)

    Am I missing your point?

  3. "Q" the Enchanter
    September 26th, 2006 @ 11:03 am

    Agree that the “problem of induction” is a philosophical rather than scientific problem. Not sure I follow the rest of the argument. The fact that individual trading decisions are explicable in terms of reasons says nothing about whether the overall picture is random. For instance, we presume that ensembles of atomic events are random even though the individual atomic careers are explicable in terms of certain forces and laws. (Indeed, we consider the individual atomic careers themselves to be random, and despite their in principle explicability.)

    Am I missing your point?

  4. JUST_ANOTHER_PRIMATE
    September 26th, 2006 @ 11:33 am

    Q asks: Am I missing your point?

    Why would you ask Raving Anitabortionist a question. He merely posts something and then lets the dust settle amongst his (dwindling) readership.

    The pope has the muslim world in an uproar, the ACLJ is fighting the ACLU tooth and nail on some very interesting religious lawsuits — lots of religious news out there and this guy is posting some esoteric mumbojumbo about stock market randomness.

    This has become a strange blog indeed.

  5. a different tim
    September 26th, 2006 @ 12:34 pm

    Why does RA accept this argument for determinism in markets but not seem to accept it in the infamous pig personality post of a couple of weeks back?

  6. Professor Chaos
    September 26th, 2006 @ 12:43 pm

    zzzzzzzzzzzzzzzzzz

  7. Andy Holland
    September 26th, 2006 @ 1:07 pm

    If the basic underlying assumption of science is as expressed; “it’s simply the underlying assumption that the same causes will always produce the same effects” then science is in for a rude awakening.

    In fact, with Poincare, science has reached very real hard mathematical limits. Poincare instability in differential equations that pop up all over the place in dynamics, means there are hard limits to what can be “objectively” analyzed, even for macroscopic systems.

    There are a host of differential equations that accurately describe behavior in the abstract, and they are fundamentally unstable in fact; which means that science fundamentally cannot count on repetition.

    You can’t measure to infinite precision, so you can’t repeat even planetary orbits for more than 2 body problems forever. These fundamental effects are worse in fluid dynamics and fluid heat transfer. You can analyze even macroscopic stuff statistically, but not deterministically.

    Unfortunately those in soft sciences don’t realize that as they deal with many, many degrees of freedom for phenomenon, determinism evaporates very quickly. It doesn’t even exist in 1D, and these effects are worse in 2D and often intractible in 3D and 4D.

    That is precisely why people were so struck with Jesus calming the wind and waves by his word as man. Theoretically its possible, if the person speaks something that compels a system to order.

    Its a shame people spend so much time on the science without a general disclaimer about these very real, very hard limits.

  8. scrotum
    September 26th, 2006 @ 1:24 pm

    J.A. Primate and Professor Chaos:

    Ha Ha! RA made you look….and post a comment….again! Weak minded fools you are.

  9. Professor Chaos
    September 26th, 2006 @ 2:40 pm

    Someone who refers to himself as “scrotum,” and constructs sentences like Yoda is going to call another person “weak-minded?” Hmmm.

  10. Professor Chaos
    September 26th, 2006 @ 2:41 pm

    RA should take these as compliments, anyways. If he weren’t “once-great,” he wouldn’t receive this treatment.

  11. Choobus
    September 26th, 2006 @ 3:36 pm

    RA is himself a living example of the dangers of induction. We got used to reading interesting and hilarious posts by him. We thought they would continue. They did not. Now Shitlords roam free as though their fecality were the norm, and we rationals the interlopers. Like the sacking of Rome by the visigoths it is. Oy!

  12. Axolotl
    September 26th, 2006 @ 3:58 pm

    Reply to Andy Holland:

    Yes, you are correct. There is no logical basis to assume that in this universe that the same cause will always result in the same result (as Hume pointed out some time ago). However, in a strickly practical sense, effect follows cause often enough in this reality for us to “assume” it.

    It is “theoretically” possible that Jesus calmed the waters by his word. It is also “theoretically” possible that God and Jesus did/do not exist, that the Bible is a collection of legends and fables, and that someday monkees will fly out of my ass.

  13. Professor Chaos
    September 26th, 2006 @ 7:46 pm

    Wow, “monkees” flying out of your ass would be a lot more painful than monkeys.

    Here’s hoping they won’t have their instruments with them!

  14. Choobus
    September 26th, 2006 @ 8:19 pm

    why would trhey have their instruments? It’s not like they know how to play them

  15. Jimmy Walker
    September 26th, 2006 @ 11:40 pm

    DYNOMITE!

  16. a different tim
    September 27th, 2006 @ 4:45 am

    I think, AH, that there is adifference between philosophical determinism, which applies to non quantum systems, and the kind of determinism in practice that you’re talking about. Noone argues that we can predict systems which are sensitively dependent on initial condidions, such as three body problems, except in abstract terms. On the other hand, nobody argues that such systems are not, at root, deterministic, which is why it’s called “deterministic chaos”.

    I think you’re muddying the waters between the scientific assumption that the same laws will apply in the same way to similar phenomena, and the limits of determining the detailed effects of those laws in practice.

  17. Paul
    September 27th, 2006 @ 8:21 am

    I don’t see why it is an *assumption* of science that the same conditions will cause the same effects in the future that were seen in the past as much as it is. Science doesn’t so much say “this will happen” as much as “every time we have seen X, Y happens.” That sentence talks about the past as an empirical fact, not a philosophic assumption. One could expand that sentence by saying “So, when you see X, bet your money on Y.” Y in that case is a prediction, not an assumption, even if the prediction is 100% accurate.

  18. Paul
    September 27th, 2006 @ 8:23 am

    Oops, delete “as much as it is” from the first sentence above.

  19. Forrest Cavalier
    September 27th, 2006 @ 10:57 pm

    that there are impediments to predictability does not mean that there is randomness.

    No, the randomness is demonstrated other ways. Take the advice of other commenters and read up on quantuum physics.

    But… admitting “impediments to predictability” lets faith in the door just as well as randomness would.

    Either way you admit you must make approximations and use predictive models in order to make good choices.

    That requires faith: trust in the existence of something for which there is no proof. You cannot prove an approximate decision-making model is correct. (If you could, it would not be an approximate model.)

    Some might even say that this predictive model or “world-view” that we all have does not even exist in fact, but is only in our mind. Regardless, it seems clear that it is informed and developed by reason, logic, and observation, but it includes many many shortcuts, guesses, and hopes which can never be proven correct.

    Further, not only is it not provably correct, some of the most important and useful beliefs of this worldview are demonstrable falsehoods such as “I am important” and “What I choose matters.”

    In a universe random and only approximately predictable, it is impossible to have free will and have no faith. (This holds for atheists too.)

  20. Godthorn
    September 28th, 2006 @ 12:29 am

    Forrest C, you impress me more and more.

    You are right about the impossibility of having free will and also having no faith.

    But. . . it is impossible in any case to have no faith.

    And. . . it is impossible that you and I possess “free will.”

  21. Forrest Cavalier
    September 28th, 2006 @ 1:42 am

    Godthorn,

    Every person’s first article of faith is that we possess free will.

    It can’t be proven that we make choices, but we believe it and act to make choices as if it were proven.

    RA’s musings about randomness come full circle here.

    We look at our personal past choices and find them to be uncaused, under our control alone.

    But “uncaused” means “random” means “free”!!!

    Godthorn, we are both right when you claim no free will and I claim there is free will. You are right by making no choice. And I am right by making one.

    In another way of looking at it: you are right because the quantuum wave function expands. I am right because the quantuum wave function collapses.

    Faith is not hiding in the gaps. It is built into the fundamental fabric of the universe.

  22. Godthorn
    September 28th, 2006 @ 2:50 am

    I have been waiting for some free-roaming shitlord or fucktard to talk back to Choobus. The only intelligent one I recall ever rebuking him is Lily, whose rebukes are nevertheless always tempered with a note of respect. I haven’t been posting here for very long, so I don’t know whether her respect is personal or merely Christian tolerance. It could be that everyone else here knows something that I do not, such as that Choobus got his balls shot off in Vietnam, or is dying of bitches or leukemia, or is blind in one eye and can’t see very well with the other–in which case a degree of tolerance is not out of order. But if some such is not the case, I am curious to know how and why he became elevated to the position of Motherfucking Superior of shitlords and fucktards. (Not that there’s anything wrong with that–nor, praise whatever god, that I am in any degree envious of the title.)

    Is Choobus an object of pity, or is he a Yoda? Since I have no clue, the odds from my perspective are that he is neither. So I am going to attempt to test one of the possibilities, providing he is willing to cooperate.

    Choobus, which came first, the chicken or the egg?

    Choobus, when a lawn is at its lushest, what color is the grass?

    Choobus, how many humans would comprise the ideal set?

    They are simple questions, and there is a definite and logical answer for each. Should you deign to answer them, I trust you will vouchafe us a precis of your reasoning, and not cheat us with a mere “this” or “that.” Your usual postings, though pungent, are suspiciously brief.

    Choobus, I half suspect you can answer all three questions brilliantly. I really don’t question your perspicacity–though I have encountered many a fellow traveler who was discerning but inexplicably limited.

    Still, I’m confident the first two questions will cause you no great difficulty. The third also will probably not, since the question implies its answer. Its explication, however, will require more
    thought, and a more meticulous rendering. So try to get back to us within a couple of days on the first two, but take as long as you like with the third.

  23. Godthorn
    September 28th, 2006 @ 4:36 am

    Forest C, I can tell you’re a nice fellow–a damned site nicer than that “Choobis” character–and you’re intelligent, too, I’ve now discovered–but you’ve been seriousely overburdened with a well-intentioned load of equine excrement.

    Your opening statement above is dead on. “Free will” is indeed our “first article of faith.” It is the first article of faith as well for all gorillas, chimpanzees, elephants, lemurs, cats, dogs, turtles and millipedes (allowing the assumption that millipedes and turtles are also self-conscious). And without going into the details of the matter–as I have done quite thoroughly elsewhere–I beg you to contemplate this: If you were able to query a chimp or a turtle on the matter, what do you think the subject would reply? Think diligently. Do you actually imagine that a turtle would answer, “I know what I want to do, but I have no idea what might be the nature or purpose of that mysterious Thing that causes me to desire it.” Read that again, my friend. Is the turtle in a state of confusion about what he himself wants, as contrasted to what he suspects may have been dictated by chance, or evolution, or perhaps even some “god”? Is the turtle suspicious of his “free will”? Does he suspect it may be a chimera, or a devine deception? No–no more than do you or I. The turtle “knows” he has free will!

    Forest, I do not “make no choice.” Nor do you freely “choose.” We both obey necessity, but because we are aware of what necessity dictates, we have the impression that we are doing the dictating.

    And faith was not built into the “fabric of the universe,” and it sure as hell is not hiding. But I have a lot of faith in it’s power.

  24. Godthorn
    September 28th, 2006 @ 5:00 am

    Damn! I wrote a passionate and extended epistle to Forest C. and the damned thing was rejected because I was posting too frequently! If no one else is posting, why can I not get another hundred words in?

  25. Godthorn
    September 28th, 2006 @ 5:09 am

    Well, as soon as I complain, I find my comment has been posted. I apologize for the complaint, RA–or whoever merits.

  26. Forrest Cavalier
    September 28th, 2006 @ 3:02 pm

    Godthorn, it is hard to tell if you were disagreeing. Self-conscious is equivalent to free will, isn’t it?

    And faith was not built into the “fabric of the universe,”

    Faith: the belief in the unproven.

    Is it a wave or a particle? The observer chooses. The observer cannot measure to determine which.

    Quantuum mechanics holds that until the observer chooses, there is no reality, only faith that there will be reality that is determined by the observation.

    Look for a wave, you prove a wave. Look for a particle, you prove a particle. Look for a seemless environment, you prove a whole. Look for a river, you prove a river. Look for a choice, you prove a choice. Look for determinism, you prove determinism.

    It’s both/and, not either/or. Built right into the fabric of the universe.

  27. Thorngod
    September 28th, 2006 @ 4:25 pm

    No, no, no. “Free will” is not equivalent to or a requisite of consciousness.

    Faith in the probable = reasonable faith.
    Faith in the unevidenced = blind faith.

  28. Marcus
    September 28th, 2006 @ 10:32 pm

    I say consciousness + control = free will.

    Me being self-aware, and my awareness being a mechanism that receives input and decides how to act upon it = free will. The fact that my awareness and choices have physical causations doesn’t seem to change that.

    So a simple lever could have free will, except that it lacks consciousness, and someone who’s paralyzed could have free will except that they’re paralyzed. As long as the self-awareness is part of the mechanism, though, and not outside of the mechanism, then it seems to me you have free will.

  29. Thorngod
    September 29th, 2006 @ 10:55 am

    Consciousness + FREE control would = free will. The “free” is the sticking point. You are who and what you must be, and if you could trace the causes of every effect on your developing self, you would search in vain for an instance in which you did not behave as you had to. You are not a free spirit directing a body; you are essentially a needy body compelling your thoughts.

  • Basic Assumptions

    First, there is a God.

    Continue Reading...

  • Search

  • Quote of the Day

    • Fifty Random Links

      See them all on the links page.

      • No Blogroll Links