The yearly intellectual progress report is a little late this year due to having to write up a PhD. Now that the PhD thesis beast is at least somewhat tamed, in a breather, I can try to figure out what progress I have made this year.
[Read More]
How the Tails Come Apart
To play with the code directly, please look at the github
repository
[Read More]
Walkthrough of "Hopfield Networks are all you need"
To play with the code directly, please check out the accompanying colab notebook.
[Read More]
Thoughts on the Falsifiability of the Free Energy Principle
Since the publication of the defining Particular Physics monograph, a question which has been repeatedly raised is whether the Free Energy Principle (FEP) is falsifiable. In the literature, it is argued that the FEP is merely a principle and thus supposedly immune from falsification. This point is often made with...
[Read More]
Predictive Coding as Backprop and Natural Gradients
This post explores some lesser known and recently discovered connections between predictive coding algorithms and the backpropagation of error algorithm in machine learning, as well as natural gradient methods. The material on natural gradients is novel to this post (as far as I know). Prerequisites are a general understanding of...
[Read More]