4 Our Final Century?

Key Points

• Understanding Existential Risk

  • Anthropogenic existential risk over the next two centuries may constitute a significant risk to humanity

  • The cause of reducing existential risks is very neglected and is undervalued by markets, nations, and single generations since existential security is an intergenerational global public good.

• The importance, neglectedness, and tractability (ITN) framework

  • The most important problems generally affect a lot of people, are relatively under-invested in, and can be meaningfully improved with a small amount of work.

• Thinking on the margin

  • If you're donating $1, you should give that extra $1 to the intervention that can most cost-effectively improve the world. There are many great initiatives with a very high average impact per dollar that will have a low marginal impact because they can't get the same efficiency at scale (they display "diminishing marginal returns")

• Crucial considerations

  • It can be extremely hard to figure out whether some action helps your goal or causes harm, particularly if you’re trying to influence complex social systems or the long term. This is part of why it can make sense to do a lot of analysis of the interventions you’re considering.

---

Scale - What is an Existential Risk ('x-risk')?

• Global vs Existential Catastrophe

  • What were some of the worse things or worse moral catastrophes in history? Why?

    • How do you compare their 'badness'?

    • How does it compare to an existential catastrophe? How much worse is an existential catastrophe?

  • Imagine a devastating nuclear war killing 99% of world's population. Civilization will eventually be rebuild to its former heights. Now compare this with a war killing a full 100% of world's population. The second war would be worse - but by how much worse?

• Definition of X-Risk

  • What is meant by the definition of Existential Risk 'threaten to destroy Humanity's long-term potential'?

    • What is meant by humanity? Does this imply discrimination against non-humans?

    • What is meant by potential?

      • Are you optimistic or pessimistic in respects to a positive future?

    • What is meant by destroy?

      • Does this include something else than extinction?

• How could it become real?

  • Role play: X-risk vs Humanity

    • Split the team into two groups: One being the x-risk with the intention to cause an x-catastrophe. The other group being humanity, trying to prevent an x-catastrophe from occurring

    • Scenario Nuclear War

      • Our world-community is split into two parties, both with nuclear weapon capacity. The first shot is fired, responded by the other side.

    • Scenario Engineered Pandemic

      • A group has developed an engineered pandemic and started it's spread.

  • What are examples of existential risks beside climate change, and how do they actually cause an existential catastrophe?

  • What are examples of existential catastrophes different to extinction?

• Probability

  • How likely do you think x-risk is to occur this century?

  • Which x-risk do you think is most likely this century?

  • What do you think about the probability given by Toby Ord of 17% likelihood this century?

Neglect - Are x-risks neglected?

• Is there neglect or actually non-neglect for x-risks?

  • In what ways has your environment (society, culture, media, education, religion, etc.) made you aware of x-risks?

  • What x-risks are you or society aware of?

  • What x-risks are you or society not aware of?

• What are the possible reasons for neglect of some and non-neglect of other x-risks?

  • Why do you think climate-change is less neglected in comparison to e.g., AI Safety?

• Why shouldn't we divert our resources into solving x-risks?

Prioritization - Which area or existential risk to focus on? ITN

Exercise: Choose a cause area, preferably an existential risk and apply the simplified ITN framework to it

• Importance (Scale) - if we solved the problem, how good would it be?

• Tractability (Solvability) - if we doubled the resources dedicated to solving this problem, what fraction of the problem would we expect to solve?

• Neglect - how many resources are already going towards solving this problem?

• Give each a value of 1, 2 or 3

• e.g., Global Governance

  • Importance 2 + Tractability 1 + Neglect 2 = 5

• Do you think the ITN framework is an useful tool to compare & prioritize between different cause areas?

Solvability

• What approaches or strategies do you suggest to most cost-effectively reduce x-risk?

  • On a political, public, research or personal level?

  • What are examples of current interventions?

• Do you think the biggest threats of x-risks are caused by nature or humanity? Does this matter when solving x-risk?

Taking Ideas Seriously

• What do the ideas imply, if taken seriously?

  • How could they impact your actions?

  • How could they impact society's actions?

• Do you think it is justified to take away resources from e.g., extreme poverty, animal welfare and invest them instead into prevention of existential risk?

Self-Care

• How to positively relate to the possible fact of this may be our last century or the possibility of an existential risk by it-self?