For curated introductions to the main themes and arguments developed here, see Sequences.
152 posts, newest first.
2026 26 posts
- Three Phases of Capabilities and Alignment
- History as Information Dynamics
- Algorithmic Progress is not Exogenous
- Can Individuality Survive in the Limit?
- Many Benefits Of AGI Could Still Be Realized In A Pause
- Math Capabilities could be Differentially Good for Alignment
- Whence the Fractals in our Stars?
- Architecture Research as Addressing Constraints to Scaling
- Mitigating Reward Hacking as Institutional Design
- Scaling is subtler than it seems
- Maximum Entropy Morality, Metaplastic Constitutionalism, and the Dynamic Virtues
- The Probe
- How can LLM RL Work Despite Information-Theoretic Inefficiency
- Is Endogenous Exploration Sufficient?
- Autarkic Agency Will Likely Migrate Upwards
- Baudrillard and Interiority
- Taking the Singularity Seriously
- Understanding Functional Decision Theory as Program Search
- Preliminary Notes on Colonizing the Universe
- Distillation Ain't What It Used To Be
- Thermopylae
- Thoughts on Claude Mythos
- When Does Competition Lead To Recognizable Values
- AI Monotheism vs AI Polytheism
- Two Mechanisms of Decadence
- Intellectual Progress in 2025
2025 20 posts
- Initial Quick Thoughts on Singular Learning Theory
- Addendum to Fertility, Inheritance, and the Concentration of Wealth
- The Biosingularity Alignment Problem Seems Harder than AI Alignment
- Gradual Disempowerment Might Not Be So Bad
- Space Warfare Seems Mostly Defense Dominant
- Whence Human Talents Neurobiologically?
- Continual learning explains some interesting phenomena in human memory
- Thoughts on (AI) consciousness
- Millennials as the Forever Generation
- The Limit of Prediction is not Omniscience
- Do We Want Obedience or Alignment?
- Most Algorithmic Progress is Data Progress
- Should we be behaviorist about an AI's values?
- Preliminary Thoughts on Reward Hacking
- Current neural networks are not overparametrized
- The Scaling Laws Are In Our Stars, Not Ourselves
- Why Not Sparse Hierarchical Graph Learning
- Maintaining Alignment during RSI as a Feedback Control Problem
- Review of Mind Children By Hans Moravec (1988)
- Intellectual Progress in 2024
2024 12 posts
- A Retrospective on Active Inference
- Right to Left (R2L) Integer Tokenization
- Alignment likely generalizes further than capabilities.
- Capital Ownership Will Not Prevent Human Disempowerment
- The Unconditioned Distribution of Current Open LLMs
- Alignment In The Age Of Synthetic Data
- Integer tokenization is now much less insane
- Does scaffolding help humans?
- Addendum to the Surprising Parameter Efficiency of Vision Models
- Fertility, Inheritance, and the Concentration of Wealth
- My Preliminary Thoughts on AI Safety Regulation
- Linear Attention as Iterated Hopfield Networks
2023 37 posts
- Learning Linear Representations through Implicit Subspace Selection
- Intellectual Progress in 2023
- Open source AI has been vital for alignment
- Addendum to Grokking Grokking
- Strong infohazard norms lead to predictable failure modes
- Preference Aggregation as Bayesian Inference
- Thoughts on loss landscapes and why deep learning works
- Entrepreneurs as Catalysts
- My path to prosaic alignment and open questions
- Private Equity as Legibility Arbitrage
- My rough categorization of AI risk types
- The case for removing alignment and ML research from the training data
- BCIs and the ecosystem of modular minds
- Eigenvector successor representations
- Hedonic loops and taming RL
- Predictive coding networks can perform causal and counterfactual inference
- Scaffolded LLMs as natural language computers
- Why GOFAI failed
- The singularity as cognitive decoupling
- GPUs vs Brains. Hardware and Architecture
- My PhD experience
- Deep learning models are secretly (almost) linear
- The surprising parameter-efficiency of vision models
- The computational anatomy of human values
- CEM as inference
- Thoughts on the future of Predictive Coding
- The Paperclip King, by GPT4
- LLMs confabulate not hallucinate
- Orthogonality is expensive
- Against ubiquitous alignment taxes
- Fingerprinting LLMs with their unconditioned distribution
- The solution to alignment is many not one
- Validator models. A simple approach to detecting and counteracting goodhearting
- Boxing might work but we won't use it
- Intellectual progress in 2022
- Integer tokenization is insane
- Gradient Hacking is extremely difficult.
2022 20 posts
- Creating worlds where iterative alignment succeeds
- An ML interpretation of Shard Theory
- Preventing Goodheart with homeostatic reward functions.
- Don't argmax; Distribution match
- The development of human sexuality as example of alignment
- AGI will have learnt reward models.
- Why not just stop FOOM?
- Deconfusing direct vs amortized optimization
- Intellectual progress in 2021
- Scaling laws vs individual differences
- Alignment needs empirical evidence
- Probabilities multiply in our favour for AGI containment
- Towards concrete threat models for AGI
- Empathy as a natural consequence of learnt reward models
- The ultimate limits to alignment determine the shape of the long term future
- How to evolve a brain
- The Scale of the Brain vs Machine Learning
- Understanding Overparametrized Generalization
- Grokking 'grokking'
- Clarifying Value Alignment in Predictive Processing
2021 4 posts
2020 11 posts
- Walkthrough of "Hopfield Networks are all you need"
- Thoughts on the Falsifiability of the Free Energy Principle
- Predictive Coding as Backprop and Natural Gradients
- The Schelling Point Theory of Money
- The AI AI safety problem
- Safer Value Learning through uncertainty
- Reflections on the Bitter Lesson
- Priors on the 10x Programmer
- Review of Certain to Win by Chet Richards
- Intellectual Progress in 2019
- Happy New Year
2019 6 posts
2017 16 posts
- What if the UFOs were Aliens
- A Predictive Processing Model of Autism
- Friedrich List: Addendums
- Thoughts on Recursive Self Improvement
- On Mathematics and Morality
- Thoughts on Metaethics
- Book Review: Cardinal Richeliu and the Making of Modern France
- Some Thoughts on Space War and Space Strategy
- Bayesian Optimization and Importance Sampling
- Book Review: Russian Embassies to the Georgian Kings 1589-1605
- Book Review: Catherine the Great by Robert K Massie
- Book Review: The High Frontier
- Reflections on Friedrich List's National System of Political Economy
- The High Frontier: Addendums
- Book Review: Who Paid the Pipers by Frances Stonor Sanders
- Contra David Lewis on the Superfluity of Worlds