Blocking Algorithms for Real-World Applications from Fraud Detection to LLMs

When working on the common data engineering problem of identifying which records across two different databases correspond to the same real-world entity — a process known as record linkage — one inevitably runs into issues associated with the worst-case quadratic complexity of doing pairwise comparisons. To understand where things can get computationally heavy, let’s look … More Blocking Algorithms for Real-World Applications from Fraud Detection to LLMs

AI Short Courses for Self-Paced Learning

I am often asked to suggest learning material for working professionals who are interested in (re)training themselves to work in or with AI. The following document, which I put together a little while ago, recommends a series of short foundational courses suitable for people pursuing self-paced learning on different aspects of AI. Section 6 is … More AI Short Courses for Self-Paced Learning

Integrating Data for AI : All You Ever Wanted To Know About Metadata Standards, Ontologies, Data Models, and Process Models But are Too Afraid to Ask

In the last 10 years or so, there has been a paradigm shift in how we train AI models. In particular, the best practice has shifted from Model-Centric AI, where we endless tweak model architectures and algorithms in pursuit of better performance, to what we now call Data-Centric AI, where we keep the model architecture … More Integrating Data for AI : All You Ever Wanted To Know About Metadata Standards, Ontologies, Data Models, and Process Models But are Too Afraid to Ask

What Makes a Good Question in the Age of AI

In a recent TED talk titled One Thing to Teach in the Age of AI, innovation practitioner Bryan Cassady shared a story about an exam created by one of his colleagues. The exam question was deceptively simple: “Prove you’ve learned the material in this course by giving me five good questions.” Many students struggled with that, and certainly … More What Makes a Good Question in the Age of AI

JEC Special Issue on Economic Crime in a Changing World

A significantly expanded version of a paper I shared earlier on Privacy Technologies for Financial Intelligence is now published in the Journal of Economic Criminology’s Special Issue on Economic Crime in a Changing World: Emerging Threats and Strategic Responses. The paper is open-access and written for a broad non-technical audience. It can be found here: … More JEC Special Issue on Economic Crime in a Changing World

Notes on Conformal Prediction and Testing

All my life I have been searching for simple and effective methods for constructing prediction intervals for different AI/ML models. I don’t know why I never encountered Conformal Prediction until recently, but I suppose it is better late than never. Conformal prediction is (arguably) the most elegant and practical technique for improving the robustness in … More Notes on Conformal Prediction and Testing

On the Semantics of Differential Privacy and Its Responsible Use

Differential Privacy (DP) is one of the most widely adopted formal model of privacy protection but its semantics, especially in the presence of correlated data and in the adversarial interactive setting, is still not broadly understood among data science practitioners. In this paper, we first look at how DP originated from research on database-reconstruction attacks … More On the Semantics of Differential Privacy and Its Responsible Use

Update on Social Cost of Multi-Agent Reinforcement Learning Paper

I recently released on arXiv a new version of the paper The Problem of Social Cost in Multi-Agent General Reinforcement Learning: Survey and Synthesis, which can be found at https://arxiv.org/abs/2412.02091 The new version has a more comprehensive description of the agent valuation functions in Section 4.1, as well as some variations and their problems in … More Update on Social Cost of Multi-Agent Reinforcement Learning Paper