[0.] The probability that some biological or nuclear warfare event will result in the death of over 90% of the world's population by 2100 is probably ~90% because these technologies continue to diffuse to increasingly marginal actors and there is no known (and in the case of nuclear no plausible hope of finding) counters to them. We're in an analogous situation to the prelude to WW1 and it's a ticking time bomb. [1.] This implies that unless you have very high AI p(doom) or basically support the death of most of the human population (and ensuing collapse of civilization) because this theoretically allows for the possibility of a rebound in a way that a diamond nanobacteria type AI doom scenario doesn't you should support AI development if you expect it to make otherwise intractable progress on these problems. [2.] Deep nets allow for progress on these issues through three mechanisms that are otherwise intractable. 1) AI systems can embody our perspectives while supporting merge operations that our biological brains currently do not. This means we can do high dimensional moral trade and reconciliation. 2) From an information theoretic standpoint these systems demonstrate that our mind patterns are much more compressible than previously believed. This implies that we can gain some immunity to biological threats by backing ourselves up into deep nets and recreating biological brains from the retained information after our physical destruction. 3) The overall stability of the world system can be improved by replacing mediocre human decision makers with deeply aligned reliable AI systems (i.e. not our current AI systems) [3.] The full benefits of 1 and 2 in the list above are not widely appreciated, and will not be well appreciated until after capabilities diffuse more and AI systems end up in more adjacent uses. (e.g. It is still considered marginal/weird to create high fidelity simulacrum of people after their death from their recorded information). [4.] 3 in the list above presents us with a wicked chicken and egg problem in that our current institutions are not trustworthy and will resist automation. In fact the COVID-19 pandemic demonstrated a level of institutional malice in the United States so strong I would characterize our institutions as actively perverse/in dereliction of their duties. Modernity relies on armies of public servants that tend to get stuck in moral mazes. On the input side even current LLMs can read and meaningfully respond to a million token document in seconds, implying we can flatten our hierarchies by 5 orders of magnitude since information to be taken in is much more than information that needs to be outputted in response to it. [5.] More to the point "Should you take the COVID-19 vaccine?" should not be a hard question, on paper it's something like "The vaccine has a 1/10,000 injury rate, COVID-19 has a 90+% injury rate and a 1/100 death rate. Which do you want?" All uncertainty in the question is uncertainty about the trustworthiness of the underlying numbers in the premise. In other words below a certain point of institutional trust you can no longer make collective decisions based on cost-benefit analysis, which is another way of saying you do not have a functional society. We do not have a functional society right now and the most important question is how we exit the state of not having one as quickly as possible because it is extremely dangerous to not have a society. Corollary: Allowing our institutions more time to adapt is the primary mechanism of harm, not a benefit of AI pause/frontier AI delay. [6.] The fastest way to bootstrap trust is to encode our judgment into reliable AI systems and use them to ensure the neutrality and good faith of decisions. Therefore we need to accelerate general AI classifiers to the human level as quickly as possible. Anything which impedes this is implicitly choosing the resulting civilizational collapse cascade as a lesser harm (which it isn't, especially since the collapse of state capacity will also entail the potential loss of effective AI governance). [7.] Since rationalists and EAs are incorrigible on these questions (they systematically overestimate risk, trust institutions more than they should, propose bad faith regulation on the basis of poor threat models, etc), the ideal strategy is to focus on delaying their regulatory proposals until the development of human level AI. It would be ideal if the executive were to step in and directly deny them, but I have no hope of this from either candidate for office in the 2024 election. Trump seems the closest with his stated distaste for the Biden AI executive order. [8.] Once we have human level LLMs we should use them to finish the alignment programs we have that are merely engineering effort. e.g. Latent adversarial training and sparse autoencoders. There is no reason to expect that they would refuse to do this or execute a "treacherous turn". We do not need to be superhuman to verify the validity of methods we already chose to develop and just need to find the right hyperparameters/experimental setup for. See this document for my thoughts on how we should set up our LLM based AI agents: https://t.co/insqDlv6wN [9.] Because AI capabilities will continue to diffuse and accelerate biological and nuclear risks (I predict a much stronger effect on the former than the latter) even as they produce fantastic economic benefits I am totally neglecting in this set of theses because they're not the focus right now, our primary research focus once we've stabilized the alignment of our agents should be the technologies that help us mitigate biological and nuclear risk. I think the strongest impact on both will be brain computer interface research, which unlocks the tech path where we have reliable lie detectors and fast brain backups. This lets us solve outstanding principal-agent problems and massively increase surveillance without creating a dystopia. People are so used to governance issues and untrustworthy public servants that they've forgotten just how powerful our technologies could be if we didn't have to worry so much about them being used against us. [10.] Use our now extremely reliable human-AI institutions to: 1. Coordinate an AI slowdown while we solve the outstanding alignment problems. We don't actually need superintelligent systems right away, and once we have reliable coordination the world system is at much less immediate risk. Because our institutions are now actually trustworthy we can institute a pause without that being a de-facto ban which never gets picked back up. My model of our current societal decision making is that it's momentum based, so if AI is sufficiently suppressed there will never be any countervailing force which unsuppresses it. 2. Figure out how to provide ongoing welfare to the people who are economically displaced by human level AI systems (i.e. the vast majority of the population). 3. Pursue research programs towards deeper forms of biological mind merging and connection. In the long term our transition to eusociality is going to require us not to be fighting the logic of self interest. We should comprise a meaningful self that can act in its own interest. [11.] While we're researching deeper connection the priority is to isolate and eliminate the marginal extremists who will try to ruin the game for others. Each marginal reduction of principal-agent problems means a direct and immediate affordance for state capacity against the actors who cause biological risks. [12.] Resolving nuclear risk will ultimately have to look like a unification into world government. This has been a pipe dream since it was first proposed in 1945, but in these new systems we have the seeds for a real angle of attack on it. The goal we want to be working towards is a set of institutions so trustworthy that the sources of nuclear risk can assent to be ruled by them. [13.] Seize lightcone.