Of the many questions we wish to answer using LLMs, it can be useful to distinguish between puzzles and mysteries. As Gregory Teverton explained in his many articles, a puzzle is a problem that has a definite and verifiable answer, but a mystery is one that “poses a question that has no definitive answer because … More Puzzles and Mysteries in Generative AI
I recently released on arXiv a new version of the paper The Problem of Social Cost in Multi-Agent General Reinforcement Learning: Survey and Synthesis, which can be found at https://arxiv.org/abs/2412.02091 The new version has These are all non-trivial extensions of the paper that build on recent new results in different fields and they are worth … More Update on Social Cost of Multi-Agent Reinforcement Learning Paper
I wrote the following comments very quickly on LinkedIn about 10 days ago in the middle of the DeepSeek frenzy, and the post turned out to be my second most-read post ever and people continues to look at it. I think the comments are holding up well so I am sharing them here as well, … More DeepSeek and All That
There’s been a lot written about Fourier Transforms over the years. From a physics perspective, I would recommend the Harmonics chapter in the Feynman Lectures on Physics. I also found Elan Ness-Cohn’s visual explanation of Fourier Transforms really satisfying. In the notes linked below, I have tried to give, in 5 short pages, the intuition … More Algebraic Intuitions behind Fourier Transforms
Fairness in Machine Learning is a topic that I have been wanting to better understand for a little while now, and this blog post summarises what I learned from reading the Fairness and Machine Learning (FML) book by Solon Barocas, Moritz Hardt and Arvind Narayanan available at https://fairmlbook.org these past couple of days. (The book … More A Simplistic Guide to Using Fairness Criteria in Machine Learning
While I have worked on aspects of AI safety for quite a few years now, in particular privacy technologies and confidential computing, I am a late convert on the importance of Artificial General Intelligence (AGI) safety research and did not take the problem seriously until about 1 year ago. My mindset has now changed completely … More The Problem of Social Cost in Multi-Agent Universal Reinforcement Learning
With NVIDIA seemingly steaming ahead in their latest quarterly result, Apple Intelligence receiving a lukewarm response from users, Wall Street increasingly worried about the return-on-investment from the hyperscalers’ massive capital investments, stories that CIOs are struggling to find ROI for AI, and news in the last two days that Intel and Samsung are both struggling … More Winners and Losers in the AI Commercial Landscape
As is well-known by now, the universal AI agent AIXI is made up of two key components: Solomonoff Induction for universal sequential prediction, and expectimax search for planning. There are several proposed and reasonably effective approximations of the Solomonoff Induction component using the factored, binarised Context Tree Weighting algorithm [WST95, VNHUS09] and its generalisation to … More Approximating Solomonoff Induction
Have you ever wondered why the natural exponential function shows up so frequently in mathematical inequalities? Here’s a graph of the natural exponential function. The constant e has a special place in mathematics, which is beautifully chronicled in Eli Maor’s book [M94]. The definition of e that is most useful and intuitive for our purpose … More Natural Exponential Functions in Inequalities
A key claim of differential privacy in [DR14] is that it provides “automatic neutralization of linkage attacks, including all those attempted with all past, present, and future datasets and other forms and sources of auxiliary information”. This is an important and often repeated claim — see e.g. [N17, Section E] and [PR23] — but the … More Dealing with Linkage Attacks using Differential Privacy
Mental Models 4 Life 