I recently realized that I have accumulated quite an archive of posts and that simply presenting it chronologically may not make it easy for newer readers to understand the core themes or arguments which develop over time.

These sequences organize selected posts into conceptual clusters and provide a pedagogical order and reading path.

These sequences do not include every post and some posts are repeated where they contain multiple themes.

Core conceptual path: From intelligence and values to post-AGI worlds

This sequence traces the central conceptual path of the blog. We begin by dissecting the computational structure of intelligence, optimization, and learned values. We then develop an approach to alignment based on these primitives, including how alignment might be implemented, targeted, and maintained over time in dynamic systems. Finally, we ponder how the boundaries and organization of minds may change after AGI, and how values might survive within worlds containing many competing or cooperating artificial agents.

The posts are ordered conceptually rather than chronologically.

Intelligence, optimization, and values

  1. Deconfusing Direct vs Amortized Optimization
  2. Orthogonality Is Expensive
  3. AGI Will Have Learned Reward Models
  4. The Computational Anatomy of Human Values

Alignment

  1. My Path to Prosaic Alignment and Open Questions
  2. The Solution to Alignment Is Many, Not One
  3. Alignment Likely Generalizes Further than Capabilities
  4. Maintaining Alignment During RSI as a Feedback Control Problem
  5. Do We Want Obedience or Alignment?

Post-AGI minds and civilizations

  1. The Singularity as Cognitive Decoupling
  2. The Ultimate Limits of Alignment Determine the Shape of the Long Term Future
  3. BCIs and the Ecosystem of Modular Minds
  4. AI Monotheism vs AI Polytheism
  5. When Does Competition Lead to Recognizable Values?
  6. Autarkic Agency Will Likely Migrate Upwards

Alignment

This provides an expanded and much more detailed sequence of my thinking on alignment and how it has evolved over time. We cover the foundational importance of alignment, the computational representation of values, specific alignment failure modes and potential mitigations, and then broader thoughts on AI strategy, governance, and deeper questions of outer alignment and how to align future dynamic continual learning agents, vs the static LLMs of today.

Foundations

  1. The AI AI Safety Problem
  2. The Ultimate Limits of Alignment Determine the Shape of the Long Term Future
  3. Deconfusing Direct vs Amortized Optimization
  4. Orthogonality Is Expensive
  5. Preference Aggregation as Bayesian Inference
  6. Understanding Functional Decision Theory as Program Search

The Computational Architecture of Values

  1. Empathy as a Natural Consequence of Learned Reward Models
  2. AGI Will Have Learned Reward Models
  3. Human Sexuality as an Example of Alignment
  4. An ML Interpretation of Shard Theory
  5. The Computational Anatomy of Human Values
  6. Should We Be Behaviorist About an AI’s Values?

Threat Models and Tractability

  1. Alignment Needs Empirical Evidence
  2. My Rough Categorization of AI Risk Types
  3. Probabilities Multiply in Our Favour for AGI Containment
  4. Towards Concrete Threat Models for AGI
  5. Why Not Just Stop FOOM?
  6. Gradient Hacking Is Extremely Difficult
  7. Boxing Might Work, but We Won’t Use It

Mitigations and Philosophy of Defense

  1. Safer Value Learning Through Uncertainty
  2. Don’t Argmax—Distribution Match
  3. Creating Worlds Where Iterative Alignment Succeeds
  4. The Solution to Alignment Is Many, Not One
  5. Validator Models: A Simple Approach to Detecting and Counteracting Goodharting
  6. Preventing Goodharting with Homeostatic Rewards
  7. Hedonic Loops
  8. Alignment in the Age of Synthetic Data

Thoughts on AI Governance and Strategy

  1. Against Ubiquitous Alignment Taxes
  2. The Case for Removing Alignment and ML Research from the Training Data
  3. Strong Infohazard Norms Lead to Predictable Failure Modes
  4. Open Source AI Has Been Vital for Alignment
  5. My Preliminary Thoughts on AI Safety Regulation

Post-AGI alignment, selection, and disempowerment

  1. Capital Ownership Will Not Prevent Human Disempowerment
  2. The Biosingularity Alignment Problem Seems Harder than AI Alignment
  3. AI Monotheism vs AI Polytheism
  4. When Does Competition Lead to Recognizable Values?
  5. Autarkic Agency Will Likely Migrate Upwards

Dynamic Alignment and the New Synthesis

  1. My Path to Prosaic Alignment and Open Questions
  2. Alignment Likely Generalizes Further Than Capabilities
  3. Maintaining Alignment During RSI as a Feedback Control Problem
  4. Preliminary Thoughts on Reward Hacking
  5. Do We Want Obedience or Alignment?

Contemporary AI

This sequence collects my writing on the contemporary deep-learning paradigm: why large neural networks work, what they learn, how their representations and architectures are structured, and what currently drives progress in frontier AI systems.

Why deep learning works

  1. Reflections on the Bitter Lesson
  2. Why GOFAI Failed
  3. Why Not Sparse Hierarchical Graph Learning?
  4. Thoughts on Loss Landscapes and Why Deep Learning Works
  5. Understanding Overparameterized Generalization
  6. Grokking Grokking
  7. Addendum to Grokking Grokking
  8. Initial Quick Thoughts on Singular Learning Theory

Representations, Architectures, and Parameter Efficiency

  1. Deep Learning Models Are Secretly (Almost) Linear
  2. Learning Linear Representations through Implicit Subspace Selection
  3. Linear Attention as Iterated Hopfield Networks
  4. The Surprising Parameter Efficiency of Vision Models
  5. Addendum to the Surprising Parameter Efficiency of Vision Models
  6. Integer Tokenization Is Insane
  7. Integer Tokenization Is Now Much Less Insane
  8. Right to Left Integer Tokenization

Language Models and Scaffolded Systems

  1. LLMs Confabulate, Not Hallucinate
  2. Fingerprinting LLMs with Their Unconditioned Distribution
  3. The Unconditioned Distribution of Current Open LLMs
  4. Scaffolded LLMs as Natural-Language Computers
  5. Does Scaffolding Help Humans?

Scaling, Data, and the Drivers of Progress

  1. The Scaling Laws Are in Our Stars, Not Ourselves
  2. Current Neural Networks Are Not Overparameterized
  3. The Limit of Prediction Is Not Omniscience
  4. Alignment in the Age of Synthetic Data
  5. Most Algorithmic Progress Is Data Progress
  6. Distillation Ain’t What It Used to Be
  7. Thoughts on Claude Mythos

Intelligence in Brains and Machines

This sequence tries to draw together evidence on the nature of intelligence by comparing and contrasting deep learning systems and biological brains, to see what each can inform about the other.

  1. How to Evolve a Brain
  2. The Scale of the Brain vs Machine Learning
  3. GPUs vs Brains: Hardware and Architectures
  4. Scaling Laws vs Individual Differences
  5. Deep Learning Models are Secretly (Almost) Linear
  6. Why Not Sparse-Hierarchical-Graph-Learning
  7. Thoughts on AI Consciousness
  8. Whence Human Talents Neurobiologically
  9. Continual Learning Explains Some Interesting Phenomena in Human Memory

The Longest Term

How can we use our understanding of the limits of intelligence, alignment, and physics to map out the structure of the long-term future?

  1. The Ultimate Limits of Alignment Determine the Shape of the Long Term Future
  2. The Singularity as Cognitive Decoupling
  3. BCIs and the Ecosystem of Modular Minds
  4. Space Warfare Seems Mostly Defense Dominant
  5. Preliminary Notes on Colonizing the Universe
  6. Autarkic Agency Will Likely Migrate Upwards

Economics, Society, and AGI

How do societies and economies work and how might AGI change their core underlying dynamics?

  1. Fertility, Inheritance, and the Concentration of Wealth
  2. Addendum to Fertility, Inheritance, and the Concentration of Wealth
  3. Millennials as the Forever Generation
  4. Two Mechanisms of Decadence
  5. Capital Ownership Will Not Prevent Human Disempowerment
  6. Gradual Disempowerment Might Not Be So Bad
  7. AI Monotheism vs AI Polytheism
  8. When Does Competition Lead to Recognizable Values?
  9. Baudrillard and Interiority

Predictive Coding and Active Inference

This sequence covers my thoughts originating from my PhD and postdoc work on Predictive Coding and Active Inference and my later retrospectives on this work. I would recommend starting from the active inference retrospective then reading backwards if interested.

  1. Predictive Coding as Backpropagation and Natural Gradients
  2. Thoughts on the Falsifiability of the Free Energy Principle
  3. Clarifying Value Alignment in Predictive Processing
  4. Thoughts on the Future of Predictive Coding
  5. Predictive Coding Networks Can Perform Causal and Counterfactual Inference
  6. A Retrospective on Active Inference

Personal

These posts form a loose intellectual autobiography. Together they record how my interests, beliefs, research, career, and writing have changed over time.

Academic and intellectual development

  1. Intellectual Progress in 2019
  2. Intellectual Progress in 2020
  3. Intellectual Progress in 2021
  4. My PhD Experience
  5. Intellectual Progress in 2022
  6. Intellectual Progress in 2023
  7. Intellectual Progress in 2024
  8. Intellectual Progress in 2025

Poetry and literary writing

  1. Thermopylae
  2. Taking the Singularity Seriously