Unifying logic and probability is an active and ongoing research topic of great interest to many. There are many proposals of probabilistic logics in the literature, each with a different motivation, either computational or philosophical, and a different system of syntax and semantics. This state of affairs is confusing and not satisfactory, especially in view … More Unifying Logic and Probability for Learning
When I was building the Data Science Centre of Excellence at Reliance Industries, I once interviewed a world-class researcher who has worked at multiple institutions, including Yahoo and Google, and he told me something I didn’t understand till then. I thought a company like Google has a weak business moat because it is vulnerable to … More The Competitive Moat of Google, Facebook and Other Data Owners
Here is a short (and somewhat unusual) course on statistical machine learning that I have delivered multiple times over the last few years. Introduction to Statistical Learning Theory Bayesian Probability Theory Sequence Prediction and Data Compression Bayesian Networks In designing this course, I have deliberately steered away from the usual practice of giving students a (long) … More A Short Course on Statistical Learning
A major deficiency in many university-level computer science programs is neglect for training in fundamental mathematical skills. This deficiency usually rears its head when a CS student first move into an area like Data Science and quickly realise s/he does not even have the ability to fully understand papers and books in the field, let alone contribute … More How to Prove It
Colleagues and friends often ask me for book recommendations on data science. Here are seven books that teach some of the most important mental models on the limits of predictability and model building, as well as prediction techniques that actually work in practice. An Introduction to Probability and Inductive Logic by Ian Hacking Against the Gods: … More A Reading List for Data Scientists
I have been studying and experimenting with online learning algorithms for support vector machines (SVMs) for a while now, primarily with the intention of understanding how they can be used to learn SVM models on large multi-terabyte datasets. The following technical report describes the NORMA and PEGASOS family of algorithms and give some observations and relevant … More Online Support Vector Machines
A colleague recently came back from his MBA class with a useful insight: Agile practices work like a (financial) leverage in project management, magnifying good things on the way up and bad things on the way down. There are already many good books and articles on the benefits of agile software engineering and agile data science so … More Agile Data Science – A Prerequisite
In this set of notes, I derive the common Bayesian estimators for Bernoulli distributions like the Laplace estimator and the Krichevsky-Trofimov estimator. Notes on Bayesian Estimators for Bernoulli Distributions
Partial Least Squares (PLS) is a widely used technique in chemometrics, especially in the case where there is multi-collinearity in the set of variables; the number of variables is larger than the number of data points; and there are multiple response variables. There are many articles on PLS but the mathematical details of PLS do … More Partial Least Squares Explained
Telcos everywhere are working on initiatives to better monetise their data. For many of them, a key challenge in addressing customer requirements is lack of labelled data. For example, a customer may come along and make a request: “Tell me something about the shopping behaviour of housewives in the country”. This seemingly simple question is actually … More Quantifying the Accuracy of Business Rules
Mental Models 4 Life 