Dunning-Kruger Effect

A strong and prevalent cognitive bias that causes a large majority of people to rate themselves more highly and more skilled than statistically possible. Lack of self-awareness can cause us to overestimate our knowledge or ability in a given area, and this phenomenon is known as the Dunning-Kruger Effect.

Posited in 1999 by two Cornell psychologists, Professors Dunning and Kruger also found that low-skilled people often have a double bind: they think of themselves as very skilled, but the lack even the basic level of skill that would allow them to detect and learn from their mistakes to get better. It’s very difficult for them to get out of the “trap” of perceiving themselves as superior, thus obviating any need to continue effort at improvements.

They also found that individuals of high skill levels also suffer from a sort of “lensing effect” (now dubbed the Dunning-Kruger Effect accordingly) in terms of their own self-assessment, but in the other direction — they are not generally aware of the rarity of their gifts. They assume most other people have the same kinds of knowledge and critical thinking skills that they do. In other words, careful study of our images of ourselves found us all to be living in a bubble of inaccurate self-perception, on both ends.

How to counteract the Dunning-Kruger Effect:

  • Ask for feedback from other people, and listen to it honestly.
  • Keep learning and gather knowledge and improving your skills.

Efficiency isn’t everything

There are many things in life you don’t want to rush through; many experiences you wish to linger. The American cult of efficiency is a kind of over-optimization, and over-fitting of a line that delusionally demands up and to the right every single day, every single quarter, every single time.

The benefits of stopping to smell the flowers have been extolled by sages and philosophers throughout the ages. In all of recorded human history lies some form of the mantra, “haste unto death” — for it is true. We rush headlong off the cliff after all the lemmings ahead of us. We can’t help ourselves — eternal moths to eternal flames.

The slow life

From the cuisine to jurisprudence, from behavior economics to psychological well-being, moving more slowly has numerous well-established benefits. Efficiency should never be the only goal, in any domain or at all times. As Madison strongly agreed with, “moderation in all things” is the mathematically optimal way to approach life, justice, and governing. Influenced by the Marquis du Condorcet, the invention of statistics, and a distaste for extremism in all forms, The Founders were prescient regarding the later theory of the wisdom of the crowds. They sought to temper the passions of the crowds via checks and balances in our system of governance.

“The arc of the moral universe is long, but it bends toward justice,” said Martin Luther King, Jr. That the veracity of the quote remains unsettled is unsettling, like strange fruit swinging in the southern breeze. Yet the “quick justice” barbaric efficiency of slavery, the Confederacy, Jim Crow, superpredators, and mowing down unarmed Black men for traffic violations to name a few, are no examples of fairness. Faster isn’t always better, especially when it comes to justice. It takes time to gather facts, talk to witnesses, piece together the crimes and document them in an airtight way, brokering no doubt in the mind of a single jurist.

More efficiency topics

Areas I’ll be further exploring:

  • Slow thinking — Daniel Kahneman’s behavioral economics and cognition theory about slow and fast thinking systems in the brain, how they physiologically arose, and their implications for bias, decision making, geopolitics, and more.
  • Journey vs. Destination — It’s not just about getting to the same restaurant and eating the same thing. The end doesn’t always justify the means. Traveler vs. Tourist. Go with the flow. Roll with it, baby.
  • An ounce of caution — A stitch of time. He who makes haste makes waste. Don’t count your chickens before they hatch. Be careful!
  • Self-reflection — Thoughtfulness. Rumination. Mindfulness. Presence.
  • Being too busy speeds up time, not necessarily in a good way. Leads to the unexamined life, a Stoic no-no. Socrates would not approve, dude.
  • Enoughness — Sustainability. Patience. Non-violence. Whole-heartedness.
  • Hierarchy vs. Fairness — Consensus takes a lot longer. Dictators and monarchs are nothing if not efficient.
  • The appeal of fascism — History and ideology of the Nazis and their obsession with efficiency.
  • PR — soundbites. Simple narratives. Tropes, slogans, repetition.
  • Entertainment — intellectual empty calories. Neil Postman. McLuhan.
  • Automation — AI, bots, robotics, threats to labor
  • Walking vs. Transportation
  • The slow food movement
  • Speed reading
  • Speed runs — video games

28 Cognitive Distortions

Sometimes our minds play tricks on us. They can convince us that untrue things are true, or vice versa.

Cognitive distortions are bad mental habits. They’re patterns of thinking that tend to be negatively slanted, inaccurate, and often repetitive.

These unhelpful ways of thinking can limit one’s ability to function and excel in the world. Cognitive distortions are linked to anxiety, depression, addiction, and eating disorders. They reinforce negative thinking loops, which tend to compound and worsen over time.

Cognitive distortionExplanationExample
all-or-nothing thinkingviewing everything in absolute and extremely polarized terms"nothing good ever happens" or "I'm always behind"
blamingfocusing on other people as source of your negative feelings, & refusing to take responsibility for changing yourself; or conversely, blaming yourself harshly for things that were out of your control
catastrophizingbelief that disaster will strike no matter what, and that what will happen will be too awful to bear"What if tragedy strikes?" "What if it happens to me?"
counterfactual thinkingA kind of mental bargaining or longing to live in the alternate timeline where one had made a different decision"If only I could have done it differently..."
dichotomous thinkingviewing events or people in all-or-nothing terms
discounting positivesclaiming that positive things you or others do are trivial, or ignoring good things that have happened to you
emotional reasoningletting feelings guide interpretation of reality; a way of judging yourself or your circumstances based on your emotions"If I feel that way, it must be true"
filteringmentally "filters out" the positive aspects of a situation while magnifying the negative aspects
fortune-tellingpredicting the future negatively
framing effectstendency for decisions to be shaped by inconsequential features of choice problems
halo effectbelief that one's success in a domain automagically qualifies them to have skills and expertise in other areas
illusory correlationtendency to perceive a relationship between two variables when no relation existshttps://en.m.wikipedia.org/wiki/Illusory_correlation
inability to disconfirmreject any evidence or arguments that might contradict negative thoughts
intuitive heuristicstendency when faced with a difficult question of answering an easier question instead, typically without noticing the substitution
just-world hypothesisbelief that good things tend to happen to good people, while bad things tend to happen to bad people
labelingassigning global negative traits to self & others; making a judgment about yourself or someone else as a person, versus seeing the behavior as something they did that doesn't define them as an individual
ludic fallacyin assessing the potential amount of risk in a system or decision, mistaking the real randomness of life for the well-defined risk of casinos
magical thinkinga way of imagining you can wish reality into existence through the sheer force of your mind. Part of a child developmental phase that not everyone grows out of.http://doctorparadox.net/essays/magical-thinking/
magnificationexaggerating the importance of flaws and problems while minimizing the impact of desirable qualities and achievements
mind readingassuming what someone is thinking w/o sufficient evidence; jumping to conclusions
negative filteringfocusing exclusively on negatives & ignoring positives
nominal realismchild development phase where names of objects aren't just symbols but intrinsic parts of the objects. Sometimes called word realism, and related to magical thinking
overgeneralizingmaking a rule or predicting globally negative patterns on the basis of single incident
projectionattributing qualities to external actors or forces that one feels within and either a) wishes to promote and have echoed back to onself, or b) eradicate or squelch from oneself by believing that the quality exists elsewhere, in others, but not in oneself
provincialismthe tendency to see things only from the point of view of those in charge of our immediate in-groups
shouldsa list of ironclad rules one lives and punishes oneself by"I should exercise more" "I should eat better"
teleological fallacyillusion that you know exactly where you're going, knew exactly where you were going in the past, & that others have succeeded in the past by knowing where they were goingacademia especially is rife with this one
what if?keep asking series of ?s on prospective events & being unsatisfied with any answers

30 Common Psychological Biases

Two psychologists ended up unlocking important keys to both the mind and to economics. Amos Tversky and Daniel Kahneman created the field of behavioral economics and revolutionized cognitive psychology with the discovery of a set of cognitive and psychological biases that affect our decision-making abilities.

These systematic errors in our thinking and logic affect our everyday choices, behaviors, and evaluations of others. For more on this topic, please also see the Cognitive Distortions and Logical Fallacies data sets.

Psychological biasExplanationExample
action biasBelief that when we're faced with an ambiguous situation or challenge, that we must take some action vs. doing nothing, whether doing something is a good idea or not (and often quickly, without taking the time to fully examine the problem); also known as "naive interventionism"sports enthusiasts rooting for their favorite teams are notorious for the superstitious rituals they are in psychological anguish if not able to perform, despite the objective fact that they have no ability whatsoever to affect the outcome (in pop culture, Robert DeNiro's character in Silver Linings Playbook exemplifies this)
adjustment heuristicTendency to start from an implicitly suggested reference point when assessing probabilities (the "anchor") and making adjustments to that reference point to reach an estimate
affect heuristicWe tend to underestimate the role of feelings of liking & disliking in our judgments and decision-makingInstead of considering risks and benefits independently, individuals with a negative attitude towards nuclear power may consider its benefits as low and risks as high, thereby leading to a more negative risk-benefit correlation than would be evident under conditions without time pressure (Finucane, Alhakami, Slovic, & Johnson, 2000)
anchoring effectFixating on a value or # that gets compared to everything else, b/c we tend to compare/contrast limited sets of items (aka β€œrelativity trap”) β€” store sale items take advantage of this (so we compare the new value to the old, but not the old value on its own as a measure of worth)
availability heuristicTendency to make quick "intuitive" judgments about the size of given categories by the ease with which particular instances/examples of the class come to mind
bandwagon effectSimilar to groupthink, arising from our built-in desire to fit in and conform, we tend to "go along with the trend" when it becomes apparent to us
contagion heuristicTendency to avoid contact with people or objects viewed as "contaminated" by previous contact with someone or something else viewed as "bad"Related to/inclusive of magical thinking β€” believing a person's sweater still carries their "essence," e.g.
confirmation biasWe tend to agree w/those who agree with us & avoid associating with those who don't, to avoid the discomfort of cognitive dissonance (the Internet has sadly made this worse)
conjunction fallacyA formal fallacy that occurs when one believes a specific condition is more probable than a general one
current moment biasPreference to experience pleasure now, & put off the β€œpain” til later; lack of ability to imagine ourselves in the future & altering today's behaviors accordingly
disjunction fallacyMisjudging that the disjunction of two events must be as likely as either of the events individually (as definitionally, via probability theory)
false consensus effectPeople tend to overestimate the degree to which the general public shares their beliefs and opinionspotentially related to the availability heuristic, the self-serving bias, and naive realism
focusing illusionPlacing too much emphasis on one aspect of an event, outweighing its importance and causing error in judgment
Gambler's fallacyPutting a tremendous amount of weight on previous events, believing they will influence future outcomes (even when outcome is random)also frequently a logical fallacy
Identifiable Victim EffectTendency for people to care deeply about a single, specific tragedy but seem disinterested in vast atrocities affecting thousands or millions of peoplemore broadly, abstract concepts motivate us less than individual cases (especially when given visual evidence)
ingroup biasOverestimating abilities and values of our immediate group & underestimating that of outgroups (oxytocin plays a role)
naive realismThe belief that each one of us sees the world objectively, while the people who disagree with us must be either uninformed or irrational"Everyone is influenced by ideology and self-interest. Except for me. I see things as they are."
negativity biasWe pay more attention to bad news
neglecting probabilityReason we're afraid to fly even though it's statistically far more likely to be in a car accident (same way we fear terrorism but not more mundane accidents that are far more likely)
observational selection biasSuddenly noticing things we didn't notice before & assuming frequency has increased (also contributes to feeling appearance of certain things or events can't be coincidence)
optimism biasTendency to believe that good things happen more often than bad things
planning fallacySystematic tendency toward unrealistic optimism about the time it takes to comple
positive expectation biasSense that our luck has to change for the better
post-purchase rationalizationMaking ourselves feel better after we make crappy decisions (aka Buyer's Stockholm Syndrome)
projection biasAssumption that most people think just like us (false consensus bias is related: thinking that others agree with us)
resemblance biasTendency to ignore statistical facts and use resemblance as a simplifying heuristic to make difficult judgments
self-serving biasTendency to evaluate ambiguous or complex information in a way that is beneficial to the speaker's interests, as well as to claim responsibility for successes and attribute failures to others or to uncontrollable external factors
shifting baseline syndromeWe tend to use very recent data points in our research (even when more data is available) and thus can miss picking up on some long-term trends
status-quo biasWe fear change, so tend to make choices that guarantee things remain the same (& by extension, assume that any other choice will be inferior, or make things worse)
treadmill effectOur desire for the new version of a product or service is acute, even if upgrades are minor & incremental; but the pleasure we get from the new object wears off quickly to leave us back at the original satisfaction baseline

No one knows more about diversity than old white dudes

For every thoughtful, measured perspective on the gigantically thorny problem of Diversity in the Valley, there has to be at least 10 angry white dudes who feel entitled to take a shit all over the idea that being more inclusive has to involve, like, actually learning to be inclusive — or really, making any changes at all.

There are “values” far more pressing than equality, they say — EFFICIENCY! ALPHA ELITISM! SHAVING OFF ANOTHER 5 MINUTES OF SOME FULL STACK ENGINEER’S TIME (by outsourcing it to someone poor who should feel lucky to have the opportunity to schlep around the dirty laundry and fetch the burritos of Today’s World-Saving Heroes — preferably someone brown) so that someone, somewhere else (outside of the Valley, one presumes) can do all the theoretical Morally Good activities that serve as the philosophical prop that is supposed to justify the tech industry’s frantic, breakneck pursuit of getting filthy fucking rich the mission critically important “time-saving efficiency” that has literally the rest of the world economy scrambling to catch up in its wake.

Ergo, in response to an interview with Slack engineer Erica Baker — whose 20% work-time role in contributing to company diversity strategy later in the thread apparently renders completely invisible her 80% role Writing Code with the Big Boys — this fellow feels he has an obligation to weigh in:

Yes, Kevin. TELL ME MORE about how I would be treated in an interview with you as hiring manager. One thing’s for sure, I could be completely confident that you lack a shred of skepticism about whether my qualifications make me “The Best” candidate in the self-fulfilling prophecy of your own perception.

Nevermind all the actual data that is finally beginning to show what the reality of nature already knows: DIVERSITY WINS. Being inclusive of a multiplicity of experience and perspective (which come along as a byproduct of the heuristic we can make use of — demographical appearance — as a rough approximate solution to our complete inability to objectively measure anything meaningful about the internal complexities of real people) makes companies stronger and more resilient.

Diversity makes companies moreantifragile by embracing the comparative disorder that is counterintuitive to the homogenous systems and societies we keep inanely trying to collectively build despite all the evidence of their abject failure throughout history. Our friend in Idaho is proof of this point: the dominant assumption that diversity definitionally reduces efficiency, thereby reducing profit.

Beyond being flat out wrong when you look at the data (which, curiously, diversity always seems to be a special case where otherwise ruthlessly data-driven engineers don’t dare to tread), this carries with it the hidden assumption which is the self-fulfilling prophecy that actually proves Erica’s point: the fundamental skepticism that people who aren’t white and male can possibly be The Best. That the only way they ever get a seat at the communal, lunch-ordered-by-bot-and-hand-delivered-by-poor-non-alpha-elite-coder-people table is by the magnanimous grace of some Do Gooder hiring manager or recruiter slavishly following regulatory orders from the government — and not by their own merit.

The plank in our own eyes

Part of this has to do with the historically definitional white male privilege that, for some reason, we’re still arguing about in our supposedly enlightened and modernized society whose blinders prevent the deep self-examination of our human past required to truly make progress. As if the human tendency to Other were somehow wiped away with the Emancipation Proclamation (1863) Fourteenth Amendment (1868) Brown v. Board of Education (1954) Civil Rights Act (1964) Voting Rights Act (1965) Loving v. Virginia (1967) Fair Housing Act (1968) Community Reinvestment Act (1977) end of theΒ carceral state (TKTK).

Having grown up a person saddled with two X chromosomes my whole life with almost no choice but to wrestle with this reality from every single angle intellectual and emotional, I at least finally understand the fundamental psychological biases that lead to this kind of abject refusal to deal with our own skewed perspectives — opting instead for ratcheting up ever more impressive shouting matches to peacock about how our dizzying intellectual prowess is surely proof enough of our obvious objectivity.

We are all wrong. And I’m no different.

I know that we desperately want to believe in our own superiority, both to everything that came before us throughout history (the “illusion of progress” we cultivate — despite no such guarantee existing in the natural world — only adds to this effect) and to our fellow humans. Elitism is the ultimate -ism.

It subsumes racism, sexism, religious fundamentalism, and all forms of tribalism that each have, at their roots, the core premise that whatever group I’ve chosen to join up with (or been allotted to by random lottery) is clearly and objectively The Best Group. It’s the undeniable tautology of naive realism that leaves us trapped in the pathetically, perennially distorted view that “I know best, and by the transitive property of awesome, all the groups I consider myself a part of are therefore clearly also The Best (else, why would I be part of them?!).” This automagically relegates all the groups with which we don’t identify to the bottom of the heap: obviously inferior, as anyone can see!

Combine this native human bias with the delirious modern cocktail of vicious neoliberalism and aggressive techno-utopian libertarianism, and it’s a formula in which People Who Don’t Appear White and Male are definitionally suspect because of the statistics we’re blanketed with ever day that tell us they are under-represented in fields like technology.

“If this is so,” says the mind of a brilliant and inarguably logical engineer, “it can only be because their Rugged Individualism hasn’t endowed them with the skills to pass muster. It’s a shame, really — at least Other People, somewhere else who care about human beings more than machine learning are concerned with this dilemma (so I don’t have to be: after all, I’m really fucking busy saving the world so STOP BOTHERING ME with this irrelevant claptrap distraction already! AND WHERE IS MY GODDAMN BURRITO?!?! It’s my Soylent off day!!!) — but honestly I have no choice but to treat The Next Brown or Curvy Data Point I See with some measure of statistical skepticism.”

Lack of diversity is a self-fulfilling prophecy

Therein lies the rub. When we take an observation about the “way things are” and leap to the moral conclusion that this is rightly so — that things ought to be this way, because clearly they are this way for some reason — we commit the logical fallacy that so consumed Hume: the idea that we can derive what ought to be from what is, also known as the fact/value problem.

I don’t think most white male engineers would go quite so far as to claim that their industry must remain homogenous to succeed (although clearly some do, like our friend Kevin, who apparently believes that diversity is definitionally both inefficient and a straight ticket to the business failure shitter — and that our only moral interest in the problem is spurred by the meddlesome interference of that old bugaboo The Government). Instead, in Silicon Valley it tends to take the form of justifying inaction: they might provisionally admit (over an artisanally-prepared, locally-sourced (from a Tenderloin window box herb garden) cocktail at Bar Crudo, or perhaps a Blue Bottle americano) that the problem of diversity may warrant some moral scrutiny, but not by them. They are just way too busy swimming for the shores of a Better World (so long as a Better World enriches them and their investors, natch) to be bothered with this issue that they perceive as not having the slightest effect on them. In times like these (which seems to be All Times), we simply can’t afford the moral luxury of anything but lifeboat ethics.

Right? Well, wrong — unless we’re not troubled by the absurd logical paradox of making ourselves subject to both the zero-sum philosophy this requires and the free market ideology of infinitely available value creation that is supposed to be driving the entire economic party bus (with karaoke) we’re riding in. So, we have to decide: which is it? Is there economic opportunity for all, or do the pathetic losers who fail to become startup founders get left at the curb? And if so, who will sing the songs of their people?!

Our own worst enemies

A reference to the old saw that “attitude is everything” is appropriate here. Because one of the few things more exasperating than the unexamined privilege of ignoring the issue is the endless infighting that those of us in marginalized groups do with each other over what the solution should be.

…where to even start? Let me explain… no, there is too much. Let me sum up: this comment from some random white dude who loves extreme sports begins and ends with the outrageously outsized entitlement of trying to tell Slack how to run its own goddamn business, from atop his lofty perch of Somewhere That Is Not Anywhere Even Remotely Near being an actual employee of Slack with some potentially arguable skin in the game, much less a leader or decision-maker within the company.

I mean, Jesus. This is what we’re dealing with. A worldview so vehemently opposed to the idea of apparently even discussing the matter of diversity (in case some terminology or phrase or godforsakenly challenging idea might be construed as controversial and somewhere, someone might possibly be offended — like the entire LGBT community he tries to lump me in with and in a follow-up comment — without a shred of irony! — attempts to claim he was only “speaking for himself” when demanding both a public apology and insinuating that Erica Baker the Slack engineer should literally lose her job for daring to state an opinion while black (p.s. we’ve truly come full fucking circle now, haven’t we?!)) that people feel compelled to spend their time offering free, unwarranted, and undoubtedly unwanted “business advice” to the company THAT PRESUMABLY KNOWS BETTER ABOUT WHAT IT IS DOING than Richard Fucking Burton The Third of His Name!

How can you even hold such a logical paradox in your head, much less lay it out in a single paragraph: the idea that somehow, bizarrely, Slack itself not only lacks the control over whether or not Erica Baker may be “let go for similar remarks” (I mean, who would be doing the firing in this case?! Is there some vigilante regulatory-required Anti-Social-Justice-Warrior in tights and a cape flying around Silicon Valley waiting for bat signals sent from comments on TechCrunch to swoop in from outside the company and authorize her termination?!), but may also be on such shaky ground from some available success metric (I assure you it’s not. It’s one of the few blindingly amazing success stories of recent memory and continues to be one of the fastest growing enterprise startups Of All Time) that they might just have to resort to taking the advice of some Totally Irrelevant Troll about what their fucking brand should be?!?

I. JUST. CAN’T. EVEN!!! (can you?! if so, better abandon all ye hope of ever working at Slack.)

Just goes to show: we’ll cling to whatever flimsy life raft of privilege we think we’re on, even as the Leaky Lifeboat (not to mention the Queen Friggin’ Mary) sails past, breathing a sigh of relief that we don’t seem eager to hop on and capsize it.

Everyone calm down. But be prepared to leave through the eastern gate

Let’s all dial down our Adderall drips for just one minute (but that’s all we can afford — the lifeboat awaits and all) and take a chill pill (feel free to take this as literally as you like). Do some soul-searching reflection, consult our Headspace apps, meditate in VR, or whatever the frak we need to do to enter the Tao Space.

Now let’s ask ourselves: if we believe we’re striving ever more harriedly toward a Better World, then what the heck does that world even look like? Close your eyes and picture it: what do you see? Are people happy in this world? Do they seem to go about their lives effortlessly and with graceful purpose in the human-connected face of god (for lack of a better term… so far), or are they still scurrying to and fro in the franticness of Trying To Get There?

Do people treat each other well, and with respect despite their differences, and in the face of overwhelming obstacles and risks we will have an impossible time solving from within isolated bunkers — or are they still spewing vitriol at each other over their gleefully intentional mischaracterizations of each other’s intentions?

Do they exhibit peace in the struggle, or are they still trying to shout each other down inside of every comment thread and social media exchange on the internet just to win a tiny provincial shadow of an urgently important argument about who has The Best Idea on how we can live in peace and harmony with each other, and how to impose it on the rest of those poor, lazy suckers who simply aren’t as gifted as the elite leaders who so grudgingly bear the wearisome heavy burden of Saving The World whilst being rewarded ever-so-handsomely with Real Non-Inflation Eaten Wages, lucrative stock options and liquidation preferences, artisanal cocktails, and Magically Appearing Burritos?

If we don’t even know what it looks like, then how will we know what values we should be working for, or recognize if and when we’ve arrived?