Title How to Actually Change Your Mind (book 2 of 2 in Rationality: AI to Zombies series)
Author Eliezer Yudkowsky
Year Published 2018
Kind of Book Rationality
How strongly I recommend it 8/10
My Impressions These are short blog posts from the LessWrong blog which were edited into a 6 part book series. Though some of the subject matter is very nerdy I always feel smarter after reading these essays. Eliezer has a way of REALLY making you question your beliefs and to put everything you think and all of your decisions under a microscope. I walk away with several new Bayesian terms and the names of a few new cognitive biases every time I read one of his books. I also walk away with more of a realization of just how irrational I am.
Date Read circa 2021
Practical Takeaways
Look for a third alternative anytime is appears that you only have two options
Before comparing your options, make sure you generate at least 3
Whenever you have only 2 options, spend 5 minutes with your eyes closed brainstorming other alternatives
Process of overcoming bias
Noticing the bias
Analyzing the bias in detail
Deciding the bias is bad
Figuring out a workaround
Implementing it
(for Scientists) instead of saying "we don't have certainty…" say, "we don't have absolute certainty…"
Argue when you disagree with people, don't light them on fire
Allow the fact that people are authorities on something to influence your belief in what they say (authorities are more likely to know counterexamples)
When you're doubting one of your most cherished beliefs, close your eyes, empty your mind, grit your teeth, and deliberately think about whatever hurts the most
(when questioning your beliefs) Ask yourself what smart people who disagree would say to your first reply, and second reply
Don't spend too much time trying to fix the thing the customer said they rejected the product because of (it might not be the real reason)
Don't promote information that you are not sure of
Do not propose solutions until the problem has been discussed as thoroughly as possible
Groom yourself well before a job interview
When trying to have a genuine open conversation on the internet, allow yourself the right to remove trolls and spam from the conversation.
When everyone comes up with a different answer than you, don't throw out your answer and jump on the bandwagon, but also take into consideration that if they are all being honest with their answer you are a lot more likely to have been the one who make a mistake.
Only do what is different from everyone else if you have a good reason to
Those who have a tendency to conform to whatever everyone else is doing must make an effort not to conform to easily (and to dissent when it is rational to)
Those who have a tendency to dissent from whatever everyone else is doing must make an effort not to dissent to easily (and to confirm when it is rational to)
When you make a mistake, say "oops" and get on with your life
If you can find within yourself the slightest shred of true uncertainty, then guard it like a forester nurturing a campfire
Admit to yourself which ideas scare you and which ideas you are attached to
Accept the uncomfortable proposition as being true, then ask yourself how you would deal with it if it were
Look for the weakest arguments for your belief, not the strongest
Big Ideas
The human ability to reason may have evolved specifically to help us win arguments
There can be a correct answer to guessing the probability of something (just like there is a correct answer to any math problem)
Our cognitive biases are part of our reasoning. It is not as is we are perfectly rational agents whose biases get in the way.
Jews (and some other religious thinkers) believe that Knowledge can only ever be lost as time goes on because the Truth was once revealed and all we have left now is the message passed down which probably gets distorted through each generation like a game of telephone
Science gains knowledge with every generation
The Torah loses knowledge with every generation
The greater the inconvenience of changing one's mind, the more effort people will expend on the proof
Most people act modest only AFTER others already know that they are awesome/cool/talented/smart etc.
We typically thinking of options once we have found one that works (rather than thinking of all the possible options that exist and then selecting the best one)
False Dilemmas are often used to force people into choosing a side or option that they don't like
The reason people play the lottery is because they enjoy the anticipation, not because they believe they will actually win
Playing the lottery is a waste of emotional energy ie. A waste of hope
We over value the importance of very very very small chances (1 in a million)
Because things aren't black and white doesn't mean they are all the same shade of gray
Everything is shades of gray
The reason Science has been such a successful enterprise is because it is built around the ability to change our minds and admit we were wrong
Moral Relativist argument for doing whatever they want
-The world isn't black and white, therefore
-Everything is gray, therefore
-No one is better than anyone else, therefore
-I can do whatever I want and you can't stop me
The more of an authority someone is on something, the more familiar with counterexamples they will be
Muddled thinking is a consequence of Muddled language
Human evil is a consequence of Muddled thinking
Knowledge of biases sometimes leads to having MORE biases
Because People with knowledge of biases have more ammunition to argue for their motivated conclusion
Not every change is an improvement, but every improvement is necessarily a change
Curiosity is a sign that someone is on the path of truth
The person who is not curious already thinks they know the answer and is working backwards to find the evidence to support it.
The person who is curious starts with the evidence and then follows it wherever it leads
Smart people only attack the beliefs they want to believe at their strongest point, rather than the weakest points
Because It is painful to think about the weak points of the belief you WANT to hold
What is true is true whether or not we acknowledge it
Acknowledging what is true doesn't make the thing that is true any worse
We can handle acknowledging what is true
Because We are already handling what is true (even if we're not acknowledging it)
The objection the prospect gives may not be the REAL objection
The more expert you are in something the most you can distinguish it for other things
Fictional narratives like books and movies are not believed to have happened, but are often still treated as historical evidence
Therefore Every piece of Utopian or Distopian fiction harms our ability to think rationally about the probability of things
We change our minds less often than we think Because The times when we changed our minds is more available than all of the times we didn't
The 911 hijackers were not cowards (It takes courage to commit suicide)
Sometimes the best response is often no response
Sometimes the reaction to something bad causes more damage than the bad thing did
The best response for America after 911 would have been to no react and rebuild the buildings
When incompetent people work together they become more confident of themselves than they would be working alone Because Each incompetent person reassures the other
There are some people who has a bias towards dissent and will dissent from what everyone else is doing regardless of whether or not it is rational
Until you admit that you were wrong, your self-image will still be bound to the mistake
Surprising Facts
Ancient Rabbis believed the worm spontaneously generated inside the apple, and therefore was part of the apple
Modern Rabbis are not allowed to overrule ancient Rabbis
The cheaper the class of objects, the more expensive a particular object will appear, given that you spend a fixed amount. eg. Buying someone a $45 scarf is considered more generous than buying them a $55 jacket.
Canadian federal elections found that attractive candidates received more than two and a half times as many votes as unattractive candidates
In one study, good grooming habits made a larger difference in getting hired than other factors
Handsome men were twice as likely to avoid prison sentences as unattractive men
People often mistake Scientists saying "we are not certain" to mean we are 50/50 on this theory rather than "we don't have absolute 100%" certainty
The reason stores have signs like 5 for $10 and other large numbers is to anchor you to a high number so that you buy more. Even if you don't buy 5 you are more likely to buy 2 than if the sign said 1 for $2
When a cult receives a major shock—a prophecy fails to come true, a moral flaw of the founder is revealed etc.—the cult members often come back with increased fanaticism rather than less because the challenge to their belief creates cognitive dissonance and they must work overtime to find reinforce their belief in the cult.
Female subjects tend to conform more than males in the Asch Conformity Experiment
Unknown Terms
Social Humility: This is the kind of humility that has to do with regulating status in a tribe. Eg. Saying you are not as smart as someone else
Scientific Humility: This is the kind of humility that deals with being scientific, double checking your calculations, and not assuming that you (or anyone else for that matters) is infallible. "it is something you would practice even if you were alone in a spacesuit, light years from Earth with no one watching."
Belief in Belief: A term coined by Daniel Dennett. This is a belief that people don't actually believe, but that they believe they should believe.
Noble Lie: a myth or untruth, often, but not invariably, of a religious nature, knowingly propagated by an elite to maintain social harmony or to advance an agenda.
The Fallacy of Gray: This is the Fallacy that says because it is not black or white it is grey and all grey is the same shade. This fallacy ignores the thousands of shades of gray that can exist in on any point. This is a move from binary thinking (where there are two options) to (where there is only one option). A move from a two-color view to a one color view.
The Sophisticate: 'The world isn't black and white. No one does pure good or pure bad. It's all gray. Therefore, no one is better than anyone else.'
The Zetet: 'Knowing only gray, you conclude that all grays are the same shade. You mock the simplicity of the two-color view, yet you replace it with a one-color view.'
(eg. No one is all good and no one is all bad, therefore no one is any better or worse than anyone else)
Sim to perfectionist Fallacy
The Monty Hall problem: On a game show, you are given the choice of three doors leading to three rooms. You know that in one room is $100,000, and the other two are empty. The host asks you to pick a door, and you pick door #1. Then the host opens door #2, revealing an empty room. Do you want to switch to door #3, or stick with door #1?
The answer depends on the algorithm. If the host always picks a door leading to an empty room, then you should switch to door #3. If the host always opens door #2 regardless of what is behind it, #1 and #3 both have 50% probabilities of containing the money.
Rationality: starts from evidence and works to a conclusion
Rationalization: starts from the conclusion and works backwards
Motivated continuation: When a person is thinking of alternatives and is not content with any of them and therefore has a desire to continue looking
Motivated Stopping: When a person is thinking of alternatives finds one they are content with and therefore stops looking for alternatives (even though there may be other and better ones)
Asch's Conformity Experiment: A study in which people were given an easy task of being asked which 2 lines were the same size. When the subject was asked first they would answer correctly, but when asked the same question after 5 people said the same wrong answer the subject would go against his or her own judgment.
Watershed moment: 1) a turning point, the exact moment that changes the direction of an activity or situation 2) a moment of humble realization, of acknowledging a fundamental problem
Fallacy of Generalization from Fictional Evidence: This is a logical fallacy where a person will use piece of fiction (TV show movie, movie, book plot) to draw conclusions about how the world actually is.
Eg. "If we do X, Y will happen because in Terminator 2 they did X and Y happened"