Integrating Data for AI : All You Ever Wanted To Know About Metadata Standards, Ontologies, Data Models, and Process Models But are Too Afraid to Ask

In the last 10 years or so, there has been a paradigm shift in how we train AI models. In particular, the best practice has shifted from Model-Centric AI, where we endless tweak model architectures and algorithms in pursuit of better performance, to what we now call Data-Centric AI, where we keep the model architecture … More Integrating Data for AI : All You Ever Wanted To Know About Metadata Standards, Ontologies, Data Models, and Process Models But are Too Afraid to Ask

What Makes a Good Question in the Age of AI

In a recent TED talk titled One Thing to Teach in the Age of AI, innovation practitioner Bryan Cassady shared a story about an exam created by one of his colleagues. The exam question was deceptively simple: “Prove you’ve learned the material in this course by giving me five good questions.” Many students struggled with that, and certainly … More What Makes a Good Question in the Age of AI

If AI really turns out to be a normal technology, what happens to industry structures?

If AI really turns out to be normal technology, an increasingly persuasive case Arvind Narayanan and Sayash Kapoor have continued to advance, what happens to the competitive dynamics and industry structures for the organisations operating in that space? Using a synthesis of Porter’s Five Forces Framework, Helmer’s Seven Powers and Oberholzer’s Value Stick that I … More If AI really turns out to be a normal technology, what happens to industry structures?

Belief Acquisition as Stochastic Filtering

We have substantially revised the factored conditional filtering paper to frame it more generally as a solution for belief acquisition in AI agents. Here’s the revised paper on arXiv. https://arxiv.org/abs/2206.02178v3 Here’s the abstract: This paper studies how belief acquisition can be accomplished using stochastic filtering. First, a theoretical foundation for empirical beliefs is outlined. Then … More Belief Acquisition as Stochastic Filtering

Martingale Tests for Model Misspecification in Bayesian Sequence Prediction

Using sequential hypothesis testing techniques to check the modelling assumptions of Bayesian mixture estimators is a promising way of getting value out of combining the Bayesian and frequentist approaches to probability. Here’s a paper to show how that can be done for Context Tree Weighting and related methods. Paper Abstract: Universal Bayesian sequence predictors like … More Martingale Tests for Model Misspecification in Bayesian Sequence Prediction

Notes on Conformal Prediction and Testing

All my life I have been searching for simple and effective methods for constructing prediction intervals for different AI/ML models. I don’t know why I never encountered Conformal Prediction until recently, but I suppose it is better late than never. Conformal prediction is (arguably) the most elegant and practical technique for improving the robustness in … More Notes on Conformal Prediction and Testing

AI Risk Assessment via Threat Modelling

Threat modelling is now considered a best practice in comprehensive technical approaches to dealing with AI safety issues [S+25]. Threat modeling [S14] is a structured, proactive process used to identify potential threats and vulnerabilities in a system. While the traditional focus is on cyber-security and privacy issues, threat modelling has been extended for AI systems … More AI Risk Assessment via Threat Modelling

Thoughts on Prompt Injection Attacks

Like many difficult cyber security problems, prompt-injection attacks is likely to become an ongoing issue that shifts and turns with the continual discovery of new attacks and new defences going forward. Instead of responding in natural language given a prompt, the best current defence I know involves always generating code, say, in a safe interpreted … More Thoughts on Prompt Injection Attacks

Customising the Australian Government’s AI Fundamentals Training Course

To support public-service agencies in the implementation of their own responsible use of AI policies, the Australian Government’s Digital Transformation Agency (DTA) has made publicly available its AI Fundamentals training course in the form of a SCORM package, a commonly used technical standard for putting together content for Learning Management Systems (LMS). The DTA training … More Customising the Australian Government’s AI Fundamentals Training Course