Week 3 - How to Reduce S-Risks?

How Can We Reduce S-Risks?

Factors influencing future outcomes

• technological progress, economic dynamics, cooperation problems, political and cultural trends

• Values of future decision-makers

Capacity building

• Face great uncertainty about what the future will look like, about how we can best influence it, and about how we can best reduce s-risks

• How?

  • building a community interested in and knowledgeable about suffering reduction

  • long-term stability of the movement

  • networking with relevant stakeholders

  • developing and refining our ideas

  • establishing healthy social norms

  • expanding our knowledge of how to best reduce suffering

Moral Circle Expansion

• Concern for suffering irrespective of

  • time

  • species

  • substrate

• How?

  • Advocacy for animal welfare reforms

  • Advancing clean meat

Suffering-focused ethics

• Moral reflection and greater clarity on our fundamental values

Politics and Governance

• Functional political system

  • Avoid harmful political and social dynamics

    • excessive political polarisation

    • Tribalism

  • strengthen democratic governance

• How?

  • voting reform

  • public service broadcasting

  • deliberative citizen's assemblies

  • compulsory voting

  • Improving norms and standards of political debate that are more charitable and thoughtful

Reducing risks from malevolent actors

• Malevolence (psychopathy, Machiavellianism, narcissim, sadism)

• How?

  • Iterated embryo selection or similar technologies

Maintaining the rule of law

• Researching how laws e.g., regarding threats and extortion can be made more robust

• Applicability of those laws in radically different contexts

• How to apply comparable regulation at level of nations, multinational companies or entirely new types of future actors

• Space Governance

  • Currently free for all i.e., potential for conflict

• Distribute future technological capacbilties

Research on bargaining and escalating conflicts

• Agents that threaten to harm other agents, either in an attempt at extortion or as part of an escalating conflict

• How?

  • research on how to best prevent such negative-sum dynamics and achieve more cooperative outcomes

    • theoretical foundations of game theory and decision theory or on finding the best ways to change the circumstances in which future agents will exist so that escalating conflicts can be avoided

Worst-case AI safety

• Technical and governance interventions to avoid escalating conflicts involving transformative AI systems, and achieve cooperative AI

Surrogate goals

• increase the likelihood that surrogate goals will be implemented successfully in future agents

  • would deflect (the disvalue resulting from) threats add to one’s current goals

  • Surrogate goal i.e., goal that one did not initially care about, in the hope that threats will target this surrogate goal rather than what one initially cared about.

Beginner’s guide to reducing s-risks

Suffering Risks

• risks of events that bring about suffering in cosmically significant amounts

• Cause

  • incidental s-risks

    • unintended consequences of pursuing large-scale goals

    • e.g.,

      • exploitation of future minds for large-scale computations needed for an interstellar civilization, detailed simulations of evolution

      • spreading wildlife throughout the universe without considering the suffering of the organisms involved.

  • agential

    • intentional harm by intelligent beings with influence over many resources

    • e.g., malevolent or retributive agents gaining control over powerful technology, or from AIs that deliberately create suffering.

  • Natural

    • processes that occur without agents’ intervention

    • e.g., future civilizations not prioritizing reducing unnecessary suffering, for reasons similar to the persistence of wild animal suffering on Earth.

• Solvability

  • researching factors that likely exacerbate these three mechanisms (especially emerging technologies, social institutions, and values)

    • Preventing lock-in states

      • e.g., deployment of powerful artificially intelligent (AI) agents, which could lead to a future shaped by goals that permit causing astronomical suffering

        • Solving the problem of alignment of AI systems with human intent does not appear to be sufficient or necessary to prevent s-risks from AI.

  • applying insights from this research (e.g., recommending principles for the safe design of artificial intelligence)

  • building the capacity of future people to prevent s-risks

    • putting future generations in a better position to make use of information they will have about impending s-risks

  • Targeted Solutions

    • research into AI designs that decrease their tendencies towards destructive conflicts or reduce near-miss risks; some forms of decision theory research; promotion of coordination between and security within AI labs; and research modeling s-risk-relevant properties of future civilizations

  • Broader Solutions

    • advocating moral norms against taking risks of large-scale suffering; promoting more stable political institutions that are conducive to compromise; and building knowledge that could be used by future actors who are in positions to prevent s-risks

Chapter 7 – Risk factors for S-Risks

Risk Factors

• What?

  • significantly increase either the probability or the severity of a very bad outcome

• Why?

  • Might not not need to know all effects that a given action will have on specific s-risks

  • If we can identify reliable risk factors, we will be able to derive robust and effective interventions for reducing a broad range of s-risks.

Types of Risk Factors

• Capacity of human civilisation to create large amounts of suffering in the first place

  • context of powerful new technologies

  • ability to create sentient artificial entities

    • exploitation of large numbers of sentient beings, due to our likely insufficient level of moral consideration for artificial minds

  • large-scale space colonisation

    • potentially multiply the total population size of both human and nonhuman beings

• Willingness but lack of capacity

  • ineffective political institutions or cooperation problems

• Indifference to s-risks or Ignorance

• Hostility

  • Disregard moral concerns

  • leave little room for prudent reflection on s-risks or for mutually beneficial compromises

  • escalating conflicts and even outright war between competing factions

• Malevolent Actors

  • Characteristics of Malevolence

    • Psychopathy

      • persistent antisocial behavior, impaired empathy, callousness, and impulsiveness

    • Narcissism

      • inflated sense of one’s importance and abilities, an excessive need for admiration, and an obsession with achieving fame or power

    • Machiavellianism

      • manipulating and deceiving others to further one’s own interests, indifference to common norms, and ruthless pursuit of power or wealth.

    • Sadism

      • tendency to derive pleasure from inflicting suffering and pain on others.

  • As a Risk factor

    • erosion of interpersonal trust and coordination

    • escalating conflicts and war

    • reckless behaviour in high-stakes situations

Complimentariness of Risk Factors

• polarisation and conflict

  • can increase the likelihood that a malevolent individual rises to power

• A dictatorship under a malevolent leader would

  • likely impede efforts to prevent s-risks

• Advanced technology

  • potentially multiply the harm caused by malevolent individuals