Week 6: Biosecurity
Resources
Detection: The Scientist: How an Early Warning Radar Could Prevent Future Pandemics
STAT: Op-ed on indoor air quality measures (10 minutes)
A Framework for Technical Progress on Biosecurity (10 minutes)
Podcast for the week: Alison Young on how top labs have jeopardized public health with repeated biosafety failures (~2 hours)
Background reading
80,000 hours biosecurity problem profile (25 minutes)
Biorisk research, strategy, and policy (20 minutes)
80,000 hours anonymous expert review series
Info hazard guidance for biosecurity discussions [BlueDot - Shared] (10-20 minutes)
Info Hazards
Generating or spreading information hazards related to biological risks can cause harm
The historical record shows that state and non-state actors have pursued the capability to cause large-scale harm with biology, and likely continue to do so.
According to an email sent by Ayman al-Zawahiri to a compatriot in 1999, al-Qaeda only became aware of and began pursuing biological weapons when “the enemy drew [al-Qaeda’s] attention to them by repeatedly expressing concerns that they can be produced simply.”
While uncovering and sharing vulnerabilities in our biosecurity infrastructure (e.g., existing medical countermeasures, detection systems) can galvanize efforts to fix them, it can also lead bad actors to take advantage of them. For example, we’re uncertain whether this disclosure about DNA synthesis screening was net-positive (probably yes).
The costs of discussing biological threat models will often be larger than the benefits.
Imagine there exists a novel bioweapon design that represents 5% of catastrophic biorisk over the next 30 years
Assuming risk increases linearly with exposure time, this could add 2.5 percentage points to the total 30-year risk of a global biological catastrophe
Potential benefits of learning about info hazards
Some threats could be very difficult to block.
Knowing about specific threats wouldn’t necessarily change our priorities.
It’s sometimes difficult to know when discussing GCBRs will be net-positive.
It’s plausible that smart, security-minded researchers who sounded the alarm about nuclear weapons in the 1940s accelerated the nuclear arms race in a harmful way
Vast majority of the brainstorming of GCBR-relevant info hazards is coming from well-intentioned discussion groups such as those in this course
Recommendations
Avoid brainstorming novel ways to cause harm with biology.
In particular, don’t create an environment where everyone is sitting around trying to figure out how to kill people with biotechnology.
Ground discussions of threats in well-known historical examples where you can.
Beyond historical examples, if it helps frame the conversation, we think one could also pretty safely consider the following threat, without speculating about the underlying biological details
Focus on broad-spectrum interventions with limited downsides.
e.g., building metagenomic detection systems to strengthening the Biological Weapons Convention
Consult others before taking actions towards reducing GCBRs.
Be an agent, not a unilateralist.
e.g., such as publishing a blog post or launching a hackathon.
Intervene kindly if discussions veer towards info hazards.
We all make mistakes, there is a lot of uncertainty here, and we need each other’s support
Use available resources, including reaching out for help.
Concrete examples that seem safe to talk about
Generally
pretty safe to discuss things that are already well known
Terrorists with the capability could in principle release multiple pandemic pathogens at once, and this is one respect in which deliberate attack scenarios may differ from natural spillover scenarios in some ways (though probably not in other ways)
The lethal dose of botulinum toxin
The documented mechanisms of action of various antibiotics, antifungals, and antivirals
This notable paper, one of two that sparked the 2012 “gain of function” research debate
This (in)famous paper about synthesizing the extinct horsepox virus
The researchers who inadvertently developed a mousepox virus which evaded immunity
The recent example of an AI drug discovery model being used to identify potentially lethal toxins
The general idea that you could in theory make a biological weapon that targets a specific person or group of people, e.g., using some sort of DNA-recognizing system like CRISPT
Specific mechanisms that could do this would be an info hazard
The fact that not all providers of synthesized DNA adequately screen their orders for dangerous DNA or malevolent customers , inc. the fact that journalists were able to order viral genome fragments in 2006
A list of specific companies that fail to screen their orders would be an info hazard
The list of organisms that were developed as weapons by 20th century biological weapons programs, and the federal US list of select agents and toxins
Items on the Australia Group’s export control lists
The fact that the genome of the variola virus (which causes smallpox) is freely available online
Kevin Esvelt’s work on gene drive technology and countermeasures
Current capabilities of LLMs for providing information on how to develop classic biological weapons agents, e.g., anthrax, including publicly-disclosed methods for measuring those capabilities
Information related to LLMs’ ability to generate ideas for novel biological weapons could, however, pose info hazards
How will we motivate people to defend against specific threats if no one talks about them?
In some circumstances, in order to motivate defensive work, intentionally disclosing specific threats to select people will be the right thing to do.
The case for engaging with info hazards changes when somebody is in a strong position to use that information effectively and responsibly to reduce biological risk
But figuring out whether to do so is a decision that really should not be taken lightly, and it’s unlikely a reading or discussion group is the right venue for this.
Pandemic prevention as fire-fighting (30 minutes)
Solvability
Fire
Almost disappeared as a cause of death in the developed world.
This might be possible for infectious diseases
Pandemic
Smallpox remains to this day only one of two infectious diseases to have been intentionally eradicated from nature
Smallpox was 'easy' to eradicate
No animal reservoir
Symptoms were distinctive, with no asymptomatic transmission.
Genome is made of DNA, which means it has a much slower mutation rate than many other viruses.
Strategies to Mitigate Risks
Prevention
Fire
Number one cause of fire is human behavior e.g., cooking
Pandemic
Killing of tens of millions of chickens and other birds due to outbreaks of the disease
Dual-use research of concern: research or technology that has the potential to do harm as well as good.
Accidental Leak
2007 foot-and-mouth disease (FMD) outbreak in the United Kingdom was the result of a leaky waste pipe at the joint site of a BSL-4 lab and a vaccine manufacturer.
Shortly after the vaccine site was relicensed for production, FMD virus leaked again into the site drainage system.
Detection
Fire
Fire alarms
halves risk of fire death
reduces risk of a fire spreading beyond room of origin by 71%
Pandemic
Infections may double every 2.5 days
Day 1 = 1 infection / day
Day 60 = million infections / day
Global Alarm system
Detection Tests
Specific to genome (if we know what we are looking for)
e.g., COVID PCR Test
e.g., Poliovirus is being found in wastewater across the world using PCR techniques
Universal to all genomes (if we don't know what we are looking for)
Passive Protection
Fire
Automatic sprinkler systems
Firefighters
Urban environment engineered around preventing the spread of fire
e.g., Fire hydrants
Regulations of building materials
Pandemic
Ventilation & Filters removing virus from indoor air
UVC light
PPE
Active Protection
Vaccine Development
For around half of the virus families that infect humans, we do not have a licensed vaccine for any virus within the family.
Accelerate the development & authorization timeline
Free, fast, and secure screening of synthetic DNA orders.
Sources of change
Innovators, engineers, campaigners, regulators, legislators
Proliferation
Governments seeking pandemic power
States generally have the capacity to do things when they decide it is in their interest.
e.g., North Korea developing nuclear bomb
Signatory of the BWC (Biological Weapons Convention) = Not Assured
e.g., The Soviet Union was a signatory to the BWC
Despite this they secretly continued a vast biological weapons program in flagrant contravention of international law and human decency.
Individuals/Smaller groups can access pandemic power too
People in the world who could be capable of assembling an influenza virus based on publicly available genetic sequences number in the tens of thousands.
In an act that might plausibly have caused more harm than good, researchers proved that they could reconstitute horsepox – an extinct virus similar to smallpox – for a mere $100,000.
Personal Reflections
Change biology of sentience (e.g., human species) so they are not affected by virus anymore i.e., mind-upload, robotic body, DNA-change
Emphasise 'foundational societal work' towards robots i.e., not affected by pandemics
e.g., food production, energy production is maintained during a pandemic
Acceleration of Vaccines and Safety Concerns
What data is available that shows that safety is maintained / strengthened while accelerating the vaccine development?
Western Bias
The article mentioned the Soviet Union as being a factor for bio-risk while not mentioning that the USA is continuing 'defensive' bio-weapon research while refusing to allow BWC verification measures
The Biological Weapons Convention: Challenges and Opportunities (15 minutes)
Solvability
Hindrances
Geopolitical challenges weakens international arms control & disarmament efforts
Doubt
Several states have made allegations related to biological weapons programmes
Facilitators
Tools
Confidence-building measures (CBMs), transparency, compliance and verification, and the national implementation of the BWC
Policy
Defining Biological Weapons based on 'intent' of use
'microbial or other biological agents, or toxins whatever their origin or method of production, of types and in quantities that have no justification for prophylactic, protective or other peaceful purposes'
Create Conceptual Clarity of a shared vision for Verification
What such a mechanism would seek to do and what it would not do
Developing mechanisms to review developments in science and technology
Developing a mechanism to promote international cooperation
Uneven distribution of the benefits of biotechnology
Without cooperation, other developments will be hindered
Ensuring a division of labour between the national and international levels
a division of labour is required between international activities and those undertaken domestically to implement the BWC and wider obligations
Building institutional support
National Implementation of the BWC
Guidance on biosafety and biosecurity
Wider governance measures such as codes of conduct and educational materials designed to inform scientists of their legal and normative responsibilities
Verification
Status
'Despite several years of negotiations among states over the course of the late 1990s and early 2000s, the protocol did not come to fruition and was rejected in 2001 largely due to US objections.'
First mentioned again at a BWC conference after 20 years
How?
satellites to bio forensics or open-source intelligence
Building transparency
CBMs are the primary agreed source of transparency in the BWC
“to prevent or reduce the occurrence of ambiguities, doubts, and suspicions, and in order to improve international cooperation in the field of peaceful biological activities”.
Status
Just over 50% of states parties have exchanged CBMs in the last few years
Importance
Terrorist Uses
e.g., Aum Shinrikyo, Japan
Use cases
Mass killings, assassination
sabotage; small-scale, deniable, localised attacks against a particular population group designed to kill, incapacitate, demoralise, or degrade such a segment of the population.
Personal Notes
Learning how to achieve increased cooperation would be universally helpful for many other moral aspects e.g., x-risks
What are the key hindrances to verification? Who?
d
Last updated