Threat modelling is now considered a best practice in comprehensive technical approaches to dealing with AI safety issues [S+25]. Threat modeling [S14] is a structured, proactive process used to identify potential threats and vulnerabilities in a system. While the traditional focus is on cyber-security and privacy issues, threat modelling has been extended for AI systems … More AI Risk Assessment via Threat Modelling
AI governance and AI assurance are sometimes conflated in conversations but they are not the same thing. In simple terms, In more details, the AI Governance Framework for an organisation is the strategic blueprint that defines the organisation’s overarching approach to developing, deploying, and managing AI systems. Its purpose is to define the roles, responsibilities, … More AI Governance vs AI Assurance
Like many difficult cyber security problems, prompt-injection attacks is likely to become an ongoing issue that shifts and turns with the continual discovery of new attacks and new defences going forward. Instead of responding in natural language given a prompt, the best current defence I know involves always generating code, say, in a safe interpreted … More Thoughts on Prompt Injection Attacks
To support public-service agencies in the implementation of their own responsible use of AI policies, the Australian Government’s Digital Transformation Agency (DTA) has made publicly available its AI Fundamentals training course in the form of a SCORM package, a commonly used technical standard for putting together content for Learning Management Systems (LMS). The DTA training … More Customising the Australian Government’s AI Fundamentals Training Course
Differential Privacy (DP) is one of the most widely adopted formal model of privacy protection but its semantics, especially in the presence of correlated data and in the adversarial interactive setting, is still not broadly understood among data science practitioners. In this paper, we first look at how DP originated from research on database-reconstruction attacks … More On the Semantics of Differential Privacy and Its Responsible Use
Cyber security and data security are closely related concepts that operate at different levels and provide different safeguards. Cyber security is primarily about controlling access to systems and data through different security protection mechanisms, from the physical network layer all the way to the application layer. These security mechanisms come primarily in the form of … More Data Security vs Cyber Security
I recently released on arXiv a new version of the paper The Problem of Social Cost in Multi-Agent General Reinforcement Learning: Survey and Synthesis, which can be found at https://arxiv.org/abs/2412.02091 The new version has These are all non-trivial extensions of the paper that build on recent new results in different fields and they are worth … More Update on Social Cost of Multi-Agent Reinforcement Learning Paper
Fairness in Machine Learning is a topic that I have been wanting to better understand for a little while now, and this blog post summarises what I learned from reading the Fairness and Machine Learning (FML) book by Solon Barocas, Moritz Hardt and Arvind Narayanan available at https://fairmlbook.org these past couple of days. (The book … More A Simplistic Guide to Using Fairness Criteria in Machine Learning
While I have worked on aspects of AI safety for quite a few years now, in particular privacy technologies and confidential computing, I am a late convert on the importance of Artificial General Intelligence (AGI) safety research and did not take the problem seriously until about 1 year ago. My mindset has now changed completely … More The Problem of Social Cost in Multi-Agent Universal Reinforcement Learning
Differential Privacy (DP) is, by now, the most widely adopted formal model of privacy protection used in industry [L23] and government [ABS22] but my sense is that its “semantics”, especially in the presence of correlated data and in the adversarial interactive setting, is still not broadly understood in the community, especially among practitioners. In the … More What Can Differential Privacy Actually Protect?
Mental Models 4 Life 