Incentive drives behaviour. If you have ever wondered why (brick-and-mortar) grocery retailers historically don’t pay much attention to understanding customer preferences — it’s getting better but things are still moving slowly — then you only have to look at how most of them make their money. It turns out that the main sources of supermarket profits … More Sources of Profit in Retail
Sometimes I can’t help but wonder whether data science as a discipline is still in the dark ages. From my vantage point as a practicing data scientist, I see that too many projects still rely on personal heroics for success. By and large, ours is still a discipline dominated by the need for stars and … More Agile Data Science: Applying Kanban in the Analytics Life Cycle
Most of us have heard of this little puzzle in retail analytics: Assuming we know products A and B are usually purchased together from market basket analysis, how do we place them on shelves to maximise sales? The often-stated “sophisticated” answer is that A and B should be placed far away from each other because … More A Note on Shopping Efficiency and Product Placement
Content-based recommendation engines are typically done in two steps. In the first, a user-preference model is constructed using a set of predefined features. (For example, in the context of retail, we may have features like Price Sensitivity, Promotion Sensitivity, Coupon Redeemer, Calorie Counter, Working Mums, etc.) In the second step, products are mapped, either directly … More A Text-Based and Self-Adapting Product Recommendation Engine
An interesting story I learned recently while reading the excellent book Time’s Pendulum by Jo Ellen Barnett is the history of how we came to have 24 hours in a day, 60 minutes in an hour, and 60 seconds in a minute. Apparently, the 24-hour convention began in ancient Egypt, and the sixty divisions of minutes … More How Come There Are 60 Minutes In An Hour
The Pareto Principle states that, for many events, roughly 80% of the effects come from 20% of the causes. The principle certainly finds expression in grocery sales data and the following diagram is quite typical of what one can observe. In retail parlance, the top product classes or SKUs are called the big heads and … More Big Heads and Long Tails in Retail
As the saying goes, retail is detail and to be successful in retail analytics, one needs to get many small things right. There is no one big simple idea in retail to guide us — marketing mix modelling comes close as a unifying analytical framework but there are still many facets that it doesn’t cover — … More How to Think About Retail Analytics
The book A Short History of Financial Euphoria by John Kenneth Galbraith provides a succinct description of the common denominators of financial euphoria, illustrated beautifully with actual euphoric episodes through the ages. The following are some key points to help us recognise such speculative episodes: Uniformly in all financial euphoria there is the thought that … More A Mental Model of Financial Euphoria
The book Rich Dad, Poor Dad by Robert Kiyosaki is probably the worst book of the personal-finance genre I have ever read, containing numerous bad advice and generally encouraging hubris instead of prudence in financial decision-making (in the name of entrepreneurism). I did, however, pick up one useful lesson from it: In finance, you need to … More Quality Financial Journalism: A List
Everyone with a rudimentary understanding of text analytics knows about term-frequency-inverse-document-frequency (TF-IDF) vectors. What is less known but deserve to be more widely appreciated and applied is a little related trick called Explicit Semantic Analysis (ESA) introduced by E. Gabrilovish and S. Markovitch, for which they were awarded the 2014 IJCAI-JAIR Best Paper Prize. (https://www.jair.org/bestpaper.html) ESA … More From Words to Concepts: Explicit Semantic Analysis
