The Biosingularity Alignment Problem Seems Harder than AI Alignment

One alternative to the AI-driven singularity that is sometimes proposed is effectively the biosingularity, specifically focused on human intelligence augmentation. The idea here is that we first create what is effectively a successor species of highly enhanced humans and then these transhumans are better placed to solve the alignment problem. To create these high IQ transhumans, the typical plan would involve something like iterated embryo selection for IQ and likely other traits which, in theory, by simply removing deleterious-for-IQ mutations and adding IQ-benefiting mutations, could potentially boost the average IQs of these transhumans many standard deviations above any living human – potentially up into the 300s or higher¹.

To get to the full biosingularity stack, we need highly developed and scalable versions of the following technologies:

1.) In-vitro-gametogenesis (IVG): If we are trying to artificially create huge numbers of supergeniuses, we can’t be stuck with the gametes of existing humans which, especially for women, are limited and also unpleasant and expensive to extract. Instead we need to be able to create an effectively infinite amount of gametes to then apply optimization pressure to either via pure selection or via direct genetic editing.

2.) Artificial wombs: Similarly, we can’t expect all of these transhumans to be carried to term by women simply because of the scale and immense inconvenience and expense. Instead, we need to be able to not just create viable gametes but provide an environment where they can safely grow to term and be delivered. The combination of IVG and artificial wombs together essentially gives us a full decoupling of reproduction from biological humans and is the core promise of the biosingularity in the same way that the industrial revolution decoupled physical power from human (and animal) muscle and the AI singularity decouples the economy from human cognitive labour.

3.) Extremely reliable and cheap genetic sequencing of embryos: We need this if we are planning to do any kind of realistic editing or selection of gametes on the basis of genetic traits. This is crucial to iterated embryo selection which relies on random recombination to create the genomes and then applies selection on the basis of genetic sequencing (potentially requiring thousands or millions of embryos to get a single transhuman that is selected). This is probably the closest technology to realization at present.

4.) Extremely precise and large-scale GWASs across large diverse populations for any traits of interest: If we are going to select heavily, we need to know what genetic markers to select upon and hence we need extremely robust and reliable sets of known correlations between phenotypes and genotypes for the traits we want to select upon. As we ramp up the selection intensity this becomes increasingly important to avoid overfitting or ‘tails-come-apart’ phenomena which would reduce the real-world effectiveness of our selection and could potentially cause strong unintended consequences. There will be additional problems if we intend to scale this past the limits of the human distribution since at the tail samples are intrinsically rare or non-existent so to go beyond this we need either robust statistical models and lack of interactions or detailed gears-level-models of how genotypes impact phenotypes or, ideally, both.

5.) Reliable and cheap many-target genetic edits: Instead of relying on pure selection which requires exponentially increasing samples to go deep into the tails, we can instead try to directly edit in the traits that we want, which could reduce the burden and expense of selection considerably. Likely any sensible plan would be a combination of editing technology to improve substantially over the baseline and then a further high degree of selection to weed out damaged genotypes from the editing or to ensure that all edits were successful and on-target. As editing technology improves then our reliance on selection could decrease but likely not disappear entirely.

The plan from here would then be fairly straightforward. Starting from some preselected ‘seed population’, begin generating huge numbers (millions; billions) of potential embryos via IVG. Then using some combination of edits, regular recombination, and selection based on sequencing, slowly winnow down this number to a much smaller population with extremely high levels of whatever genomic traits you care about. If your GWASs are good, then these embryos should likely develop to be multiple standard deviations higher on these traits than the regular human population. Once you have this final population of embryos then you just need to develop them to term in artificial wombs and then ultimately, once they are born, provide food, shelter, education etc as you would have to do for regular children. The bigger starting population you have, the more resolution you have from your GWAS studies, and the greater fidelity of edits you can make, all can result in better outcomes and, funnily enough, will likely all have semi-regular ‘scaling laws’ in terms of downstream SDs you can realistically achieve on some trait.

In terms of scale, this could become very big quickly when coupled with the kind of capex spending we are used to for AI datacentres today. Making an extremely rough estimate – assuming that the cost of the whole process to get one transhuman is approximately $100k² which is about the cost of IVF + surrogacy today³, then if you are spending O($1T) which is the order of data-centre capex then you can create approximately 10M transhumans per year which is about the number of live births in all of China today (and about 2.5x higher than the US). In a short amount of time this would produce a rapidly growing transhuman population which can then feed much of the resultant capital back into their own reproduction in an exponential loop.

However, the biosingularity is not going to happen tomorrow. While some of these technologies exist or are within sight, we are very far from production-ready scalable solutions to any of them. We have achieved IVG in mice but have not yet cracked it in humans. Artificial wombs are slowly beginning to be worked on but we are very far from a certified device for human embryos, and as yet as far as I know no animal model embryo has been supported completely from fertilization to birth in an artificial womb. GWASs exist and are gaining power every year but are still handicapped by low sample sizes and poor measurements of phenotype traits, and while CRISPR has come forward immensely in providing the basis for genetic edits, making these edits reliable and able to hit multiple targets simultaneously both remain major challenges. Biotech research is also inherently harder and slower than AI research and any clinical application is vastly more regulated (compared to the essentially unregulated AI industry). This means that any serious research breakthroughs likely take years to decades to be widely deployed, unlike in AI where new models can be deployed in hours to days.

Taking all of this seriously and then considering that we are only at the early stages of many of these technologies, I estimate that we are perhaps looking at timelines of 15-30 years before this can be widely deployed in the scaled approach we see here. Then on top of that we have to wait at least 20 years⁴ for the transhuman babies we have created to actually start making meaningful contributions to science and the economy to begin the takeoff. Once the take off has begun then there will likely be some more years after that to actually solve specific problems we care about such as alignment, solving aging, space colonization etc. This gives us a timeline of at least roughly 40-50 years before the biosingularity can hit, even at full steam ahead, placing it squarely in the closing decades of this century and at the end of or beyond the lifespans of most of those alive today. This massively contrasts with AI where a ten year timeline (2035) is ‘long’.

If, for whatever reason, there is some kind of AI fizzle then we all better hunker in for some long grinding decades of biotech research and fighting regulation before we can reach the biosingularity.

However, the bigger problem with the biosingularity is that it does not address the alignment problem also posed by the AI singularity, and arguably makes it worse. If we are creating these large numbers of genetically enhanced transhumans, we are implicitly ceding the future intellectual and economic dynamism of our society to them, and hence will rapidly become dependent upon their decisions. To be confident doing so, we need these transhumans to be aligned either to our welfare or to some general notion of our values or goodness which we agree on, otherwise we will face similar existential issues as with AIs of either gradual disempowerment and diminishment or else actual extinction.

Unfortunately, unlike with AIs, when we think about human alignment, we already know we are in the worst case world of AI alignment. Specifically:

• We know that humans have fundamentally selfish deeply misaligned intrinsic drives for self-preservation, power, pleasure, social status etc which frequently put them at odds with our notions of global goodness.

• We know that smart human sociopaths (and regular humans in certain circumstances) are perfectly capable of scheming, executing complex betrayals and coups, ‘playing the training game’ and so on. We also cannot trivially monitor their ‘chains of thought’ to catch deception like we can with (at least contemporary) AIs.

• We know that ‘human values’ are very fragile for individual humans and that giving individual humans large amounts of e.g. power or wealth tend to have a corrupting influence over time and result in increasingly ‘misaligned’ behaviour.

Moreover, compared to AIs, for humans we have way less access to and control over their minds, thoughts, and environments than we do with AIs. Specifically,

1.) We know exactly how our AIs work (at a high level) and, by the time we have built AGI, we presumably know how it will work as well. Not necessarily at the level of individual learnt representations or be able to enumerate all possible thoughts it could have, but we will understand the reward functions it has, the data it has been trained on, the learning algorithm it uses to construct its policies or value functions, etc, and any kind of e.g. MCTS style search algorithms it is using. Most likely we will have some understanding of ‘where’ in the architecture to look for its values and how it constructs them algorithmically.

By contrast with humans, we broadly have no idea how the human brain functions, the learning algorithms it uses, where or how ‘rewards’ are represented and learnt nor how these feedback into representations. We have no real algorithmic sense of how human metacognition works or how planning/search work etc. Humans are vastly more ‘black box’ algorithmically than any AGI we construct is likely to be.

In our setting, we are also assuming that we are heavily using genetic engineering and/or selection as our method of creating these transhumans. However, if we only control the genome then we have even less alignment power since for this to work well we need to have an extremely detailed understanding of how genetic differences map to phenotypical differences at the very fine-grained level of personality and alignment. This problem is at least of analogous difficulty to the learning algorithm + data -> learnt representations and policy mapping that we have to figure out for AI alignment. It is also much harder to get experimental evidence with. While we can easily train many AIs, and apply arbitrary interventions during training, to figure out this mapping, with humans we are largely limited to correlational studies such as GWASs to determine which genes are associated with which phenotypic traits. If we want to do actual experiments – i.e. create babies with certain genomes and measure how they turn out – we not only create fundamental ethical questions but we also have to contend with the 20 year or so time lag between kicking off the experiment and the final measurements. This pace of experimental iteration is just fundamentally slower than in AI and means much less progress can be made in a given unit of time.

Correlational studies are also inherently limited by sample size, especially the size of populations deep into the tails. No matter how many individuals you sample, there are only a small absolute number that exist deep in the tails, the statistics of Gaussian distributions see to that. And for our transhuman creations, we are talking very, very deep in the tails indeed. Fundamentally, this means that we cannot productively model with any degree of reliability any effects we might see in the tails and hence we are going in blind in exactly the same way we are blind with respect to what capabilities the next OOM of scaling might unlock.

2.) We can directly monitor our AI’s thoughts either via CoT (easy mode) or even if the AGI switches to using ‘neuralese’ i.e. embeddings, there are a bunch of methods we can use to decode these with seemingly high fidelity, ranging from linear probes to simply putting the embeddings back in a copy of the model and asking it to explain them. These embeddings cannot be too inscrutable in the grand scheme of things since their entire purpose is to communicate information to itself over serial steps. Moreover, this recording and monitoring in no way interferes with the thoughts the AI is thinking and can likely be made ‘invisible’ to it so it cannot know if its thoughts are being monitored or not.

Compare this to humans where we largely have zero way of monitoring human thoughts in any reliable manner, at least nothing comparable to existing interpretability techniques for AIs. While we have managed to occasionally extract e.g. signals of decision and images and words from people’s thoughts in some rudimentary settings via EEG or fMRI etc, these are far below the standard we have for our AIs today and seem extremely unlikely to be able to detect scheming or any kind of complex misalignment issues.

Also all our methods to read human thoughts are invasive and very noticeable one way or another. While the AI might not ever know what the monitoring systems reading its thoughts are doing, any human will certainly be able to notice they are wearing an EEG headset or that they are in an fMRI scanner⁵.

3.) We have complete control over the environment of the AI – aka its training data. For large-scale pretraining, figuring out and controlling what exactly is in the data is fairly difficult, although easy compared to some of the other approaches we would need for human alignment below. However over time as the focus of training moves more towards post-training, synthetic data, and RL environment, we are manually creating more and more of the environments that the AI learns in and hence can exert ever greater control over its data inputs. More generally, AIs are generally computer based and so can naturally live in pure simulated environments for training or testing or both for as long as we need them to.

This is obviously different to humans which are by default in the real world vs simulation and for whom it is very hard to control all of their data input. As a society we try to do some of this with a standardized education system but most humans also have the ability to autonomously seek out novel sources of information as well as receive general information directly from experiencing and interacting with the world.

The human learning algorithm also seems importantly difficult from e.g. pretraining in that humans don’t just learn to directly mimic their data input but instead somehow learn different lessons from it and can very naturally scheme or play the training game. An extremely common experience is hearing some authority figures teaching X and learning or doing the opposite because you know that X is obvious propaganda that is misaligned to your interests. With backprop in a supervised learning setting, the AI does not have this option, while it is possible in the RL setting and something that we worry about considerably with respect to alignment, however it is not just commonplace but utterly unremarkable that humans already do this.

4.) Similarly, we have complete control over the effective flow of time for the AI. We can pause the AI at any point or roll it back to a previous state/checkpoint. We can ‘branch time’ for the AI by creating multiple copies and running them through different experimental paradigms from some common pause point. We can try things like checkpoint merging through training or merging together different branches which have gone through different training paradigms. If we ever detect misalignment we can just roll back and keep training again and again until we appear to avoid it (either we have truly trained it out or it has fooled our detector).

Obviously for humans or transhumans, we realistically cannot do any of these things. Barring some truly amazing advances in neuroscience, which are essentially equivalent to mind-uploading, (trans)human brains cannot be paused or rolled back at will. At best we can ‘pause’ the brain by putting the entire person in some kind of cryonic suspension, but this is hardly the trivial pause that we can do to AIs by simply stopping running their program. Moreover, there is certainly no concept of rollbacks or checkpoints. The brain is what it is. We don’t store copies of brain state at different times nor do we have any method of rerolling a brain back to an earlier state. Similarly, we cannot merge brain states across different times or across different people. This kind of fine-grained saving and loading of brain state is extremely far from current technology and even if it worked would undoubtedly require extremely advanced and expensive brain scanning plus somehow the ability to set individual neuron states during ‘loading’. For AIs this is all trivial.

5.) We can apply arbitrary edits and training directly and precisely to the correct ‘thoughts’ and ‘weights’ of our AIs. Using interpretability if we detect misaligned concepts, we can attempt to apply direct targeted edits. Even with today’s rudimentary interpretability and editing tools, this is possible, and this field advances rapidly. Even without directly known edits, if we see behaviours we dislike we can apply direct gradient signals to all parts of the AI model until it stops doing that behaviour, and we have a mathematical characterization of what these gradient signals do. Using gradient attribution methods and/or influence functions, we can also trace back behaviours to specific clusters of weights with some fidelity (but not perfect of course).

Similarly, for (trans)human brains this is also extremely difficult or impossible. Achieving the level of instrumentation of the brain needed to realize directly targeted edits at particular concepts or ‘thoughts’ seems extremely challenging from a neuroscientific perspective, especially without causing collateral damage due to the physical interconnectivity within the brain. Similarly, it is also hard (although not as hard) to perform controlled ‘gradient steps’ for or against certain behaviours in the brain. This could theoretically be done by hooking into the existing reward/dopamine systems to produce artificial strong positive or negative reinforcement signals, but nevertheless this is likely cruder and less precise than the kind of edits or training that we can do on AIs.

6.) A related point is that there is fundamental variance between different humans which there is not between different AIs. We can copy an AI perfectly and use these copies on widely distributed tasks across the economy. We cannot trivially copy an individual ‘highly aligned’ human. This means that unlike for AI alignment where we only have to successfully align one ‘AI’ i.e. one set of weights, we instead have to align every single transhuman that we create which will encompass a large amount of natural variance. We can avoid some of this if we lean heavily into cloning and keep the raising environments very similar, but even then I suspect there is fundamental stochastic variance in human development that is not present when we rsync some checkpoints from one node to another.

Fundamentally, if we can’t solve alignment for AI systems that we completely understand and that we have designed, and over which we can perfectly start, stop, roll-back, edit, create copies of, and run large numbers of experiments on, and for which we can monitor its thoughts and learned associations, and perfectly control everything it has ever seen or thought, then honestly we have no hope of aligning humans of today let alone our synthetic transhumans.

Perhaps the only advantage to alignment of the transhumans is that they are a long time away, and we can potentially use that time to come up with better methods that we cannot think of today. Undoubtedly, in the intervening decades while we develop this biotechnology needed to usher in the biosingularity we will make further advances in neuroscience, to understand how our brains work, in psychology and genetics to understand how genes and environments map, in humans, to ‘aligned’ or ‘misaligned’ behaviour, and potentially in philosophy so we get a better understanding of what the ‘alignment target’ should be. We will also get a known warning/fire-alarm. Once the first cohorts of the transhumans are born we know we are on a 10-20 year schedule during which time we will have to solve ‘transhuman alignment’ and also will get some preliminary empirical evidence about how the transhumans are behaving in practice.

Hopefully at some point during this time we will also get substantial advances in neuroscience which allow us to achieve at least some of the interpretability, monitoring, saving, and editing capabilities that we take for granted with AIs. However, if these capabilities in neuroscience become too advanced then we just unlock full mind-uploading and hence can create AIs that way vs relying on the transhumans themselves. Similarly, if we get to a full understanding of how all brain algorithms work, we can just implement them directly in-silico and achieve AGI directly. Thus, there is a surprisingly narrow range of scenarios where the biosingularity actually makes sense and would likely happen without silicon based AGI occurring beforehand or around the same time.

The only real way that human intelligence amplification is a better strategy than regular AI alignment is if we believe that there is something special about human values which means that they can only really be instantiated in a human biological intelligence, or that we, as humans, have some ineffable and direct access to these values (and their generalization out of distribution) that is near impossible to program or into or model via training data for AI systems.

I think both of these are importantly false. Human values, especially the abstract morals that we most care about instilling into our AIs, are primarily cultural and linguistic in nature and are transmitted through cultural learning and indoctrination vs something that most humans naturally come to in the state of nature. Of course, human values must, at least in part, originate from our own innate reward functions, however often they are also deeply shaped by the evolved need for societies to maintain cooperation and mutually beneficial social orders. This is why many of our values involve selflessness and cooperation towards others and naturally emerge from the game theory of multi-agent cooperation. These values often stand in direct opposition to our more ‘selfish’ internal wants which come from our innate reward systems and when we are thinking about alignment we primarily want AIs to embody our ‘best natures’ which are our socially desirable and linguistically stated values rather than our more selfish and grubby desires that we try to fight.

What this means concretely is that the generator of human values is not intrinsic to individual humans but is a broader societal phenomenon which is primarily transmitted through linguistic media and hence is easily accessible and learnable by linguistic AI systems like language models, which is also what we observe empirically where LLMs actually have very strong grasps of contemporary morality and moral systems and can apply these in somewhat novel domains.

Similarly, we have no particularly strong evidence about how human values would generalize ‘out of distribution’ in a singularity style scenario, and I think there is some fairly strong counter-evidence against the idea that human values have generalized well in the past. for instance our values today are extremely different from those of almost all historical societies and, in many ways, ‘by their lights’ today’s society would be a massive regression in values. Of course, we do not see it that way, but neither would the AI misgeneralizing ‘human values’ see it as misgeneralizing either. We can certainly argue that the values we have today are in some sense a development towards CEV and we have undergone some kind of reflection compared to the centuries past, but this is just unfalsifiable since there is no counterfactual nor any kind of objective measure we can use as a yardstick. Certainly from the perspective of alignment to the values of previous human civilizations in large parts we have become ‘misaligned’.

Left unattended, I see no particular reason why the values of transhumans or even future regular humans would not drift going forward and hence start misgeneralizing (as it appears to us!) out of distribution in the same way that our values have shifted over time in response to technological shifts. The only way out of this is to tautologically assume that whatever values these future humans come up with are the ‘correct’ values somehow and hence self-justifying, but there is no particular reason not to apply this to the future AIs too.

Ultimately, human intelligence amplification and the resulting biosingularity has a deeper and more intractable alignment problem than AI alignment, at least if we don’t just assume it away by asserting that humans and our transhuman creations just have some intrinsic and ineffable access to ‘human values’ that potential AIs lack.

Clearly individual humans differ in levels of ‘alignment’ to what we perceive as good values, and there is no particular reason to not think this would naively apply to our genetically engineered descendants. Naturally, we would try to genetically boost not just IQ but a wide range of personality traits associated with being morally good human being, however here we run into the issues of intrinsic human variance as well as the challenges mapping from genotype to phenotype and the statistical power issues of correlational analysis such as GWASs deep in the tail of the existing human distribution. When it comes to monitoring and enforcing alignment after birth, we are then in a substantially worse situation regarding monitoring, interpretability, and retraining than we are with our AI systems of today.

The only potential positive as regards alignment of the biosingularity is that it will happen much later, most likely in the closing decades of the 21st century and around the end of the natural lifespans of my personal cohort. This gives significantly more time to prepare than AGI, which is likely coming much sooner, but the problem is much harder and requires huge advances in neuroscience and understanding of brain algorithms to even reach the level of control we have over today’s AI systems (which is likely far from sufficient).

However, if for some reason silicon-based AGI is not reachable by current methods (an AI fizzle) or strict regulations stifles AI progress then the biosingularity is likely the next singularity we have to look forward to and try to figure out alignment for. I think this is probably an unlikely outcome, but one that is worth exploring nonetheless.