Week 2: Epistemics

Reasoning transparency (30 minutes)

• Benefits of Reasoning Transparency

  1. Fciliate the process of updating other's view in response

• How?

  1. Open with a linked summary of key takeaways

  2. Indicate primary & secondary considerations for your key takeaways

  3. Indicate confidence levels

     1. 0-100%, plausible, seems

  4. Inform about concluding variables underlying confidence levels

     1. A reasoning that seems robust

     2. shallow/careful examination of one or more studies that you feel weakly/strongly qualified to assess

     3. Verfiable facts coua can provide sources for

     4. Expert opinions

     5. Vague impression

     6. e.g.,

        1. my view is based on a large number of undocumented conversations, such that I don’t think it is realistic to aim for being highly convincing in this post.

        2. It is widely believed, and seems likely, that regular, high-quality sleep is important for personal performance

        3. [CBT-I] can be delivered on an individual basis or in a group setting,

        4. As far as we know, the vast majority of human cognitive processing is unconscious, including a large amount of fairly complex, ‘sophisticated’ processing. This suggests...

  5. Inform about Crucial Considerations

  6. Inform experience, trustworthiness and fairness

Aaron’s epistemic stories (15 minutes)

• Finding Trust-worthy sources of information

  • Fact-Checking sources of information e.g., news

  • Does this journalist/news update their views?

• Attune to optimism & pessimism bias

  • Check on 'Predictions' e.g., 'next big thing', 'time-lines'

  • Align goals with reality e.g., tasks & completion time

How do you predict the future? Ask samotsvety (15 minutes)

• Benefits

  • Better Predictions = better decisions = better actions = better impact

  • Understanding the model of the world

• How?

  • Thinking in numbers > words e.g. 'unlikely'

  • Two Key Steps

    • Acknowledge that the future & past will look mostly the same

    • Isolate the small cases where the two are different

  • Practice develops the skill of forecasting

  • Updating one's forecasts

  • Base rate

    • rate at which some event tends to happen

    • e.g., NY Yankees won 27 times the world serious out of 119 i.e., 22.7% is the base rate

• Difficulty predicting the cases of outliers

  • e.g., nuclear bomb hitting London

Scott Alexander, Beware the Man of One Study (15 minutes)

• Evidence is complex

  • Due to the nature of experiments there will usually be a bell curve i.e., opposing results while finding middle ground

• Decrease your confidence about most things if you're not sure that you've investigated every piece of evidence

• Do not trust biased sources

  • e.g., websites

• When receiving 'overwhelming evidence' do a simple search to see if the opposite side has equally 'overwhelming evidence'

Michael Aird on ‘independent impressions’ vs. ‘all-things-considered beliefs’ (5 minutes)

• Independent Impressions

  • What you believe about X if you do not consider updating your beliefs despite receiving peer opinions

• All-things-considered beliefs

  • Takes into account peer opinions

• Suggestions

  • Recognize & distinguish independent impressions from all-things-considered beliefs

  • State clearly about expressing independent impressions or a-t-c beliefs

  • Feel comfortable expressing our independent impressions

  • Make decisions based on a-c-t beliefs > independent impressions

Sequence thinking versus cluster thinking (45 minutes)

Sequence Thinking

• What?

  • Decisions based on a single model of the world

  • Belief 1 -> Belief 2 -> 3 -> 4 -> Conclusion

  • If each step A, B, C, ...,N holds true, then the conclusion X logically follows

• Benefit

  • Generate (novel) ideas

    • e.g., contradict conventional wisdom

  • Deep Understanding

  • Transparency

    • Each assumption is explicit

    • Causal Understanding

• Disadvantage

  • One big factor (even if uncertain) can domiante hte final conclusion

  • Vulnerability to Errors

    • If any single assumption is flawed, the entire conclusion might be flawed

    • Overconfidence/Underconfidence is not cushioned i.e., will have a strong impact on one's conclusion

    • Facilitates 'end justifies the means'

    • Facilitates non-exploration of crucial considerations

Cluster Thinking

• What?

  • Decisions based on multiple models of the world

  • Weighing of all: Perspective 1 implies X, Perspective 2 implies Y, Perspective 3 Implies X

  • “I think we should do X, but I can’t say exactly why, and some of the most likely positive outcomes of this action may be outcomes I haven’t explicitly thought of.”

  • Cushions high weights

    • De-weighing of single beliefs/models even if high expected value

• Benefit

  • Making final decisions as it is more balanced

  • Robustness

    • Avoids the potential wrongness of a single-model world

• Disadvantage

  • Complexity

    • May lack clarify of a single-model approach

    • Weighing & integrating various perspectives might be challenging

  • Conflict

    • Diverse view points might lead to conflicting conclusions

How To Think?

• Robustness & uncertainty

  • probability that my view remains stable despite gaining more information

• Regression to normality

  • Unusual claims require strong evidence to make a shift

• How to Think?

  • Combination of Sequence & Cluster Thinking

    • Sequence to explore possibilities & cluster to evaluate them comprehensively

  • Well-understood situation: sequence > cluster

  • Complex / less understood situation: cluster > sequence

  • Recognize advantages / disadvantages of each

Personal Reflections

• World as a cluster

  • Every individual uses sequence thinking i.e., 'belief 1 -> believe 2' even if that includes cluster thinking i.e., they come out with one conclusion

  • As a world, we share our sequence thinking conclusions and come to harmony i.e., cluster thinking

Accurately predicting the future is central to absolutely everything. Phil Tetlock has spent 40 years studying how to do it better. (2 hours)

Benefits of Forecasting

• Probabilities guide our decisions and our behavior

  • Lack of probabilities hence may lead to mistakes e.g., 'impossible', 'possible', 'certain'

• Superforecasters demonstrate the ability to make reliable forecasts even for 'unique' events

  • e.g., Brexit, Russia-Ukraine

• World would be better off if government leaders would consult technocratically run forecasting tournaments for estimates on key issues

Hindrances to Forecasting with Probabilities

• In a polarized world, one makes the claim that one is only e.g., 75% convinced of one side - one could face detrimental consequences

Caution

• Counterfactual Reasoning is ideologically self-serving

  • Conservatives

    • Without President Reagan, Cold War would have continued and Soviets would've pushed further

  • Liberals

    • Soviet was economically collapsing and Reagan wasted billions in unnecessary defense expenditures

• Claims imply counterfactual claims

  • e.g., Every time you claim someone to be a 'good' or 'bad' president you imply the claim about hwo the world would have unfolded in an alternative universe to which you have no empricial access

How?

• Willingness to Update

• Probabilistic Thinking

  • Discriminate between differences as fine as 56% versus 57%

• Macro into Micro

• Acknowledge Confidence (Over/Under)