Tag: Greenplum

How to Link Millions of Addresses with Ten Lines of Code in Ten Minutes

July 16, 2017

Solving big hairy problems like detecting complex financial crimes requires solving a series of smaller, mundane but technically non-trivial problems. Performing efficient record linkage on large databases with tens to hundreds of millions of rows of data is one such pesky problem. A few of my colleagues have just made a small dent on the overall … More How to Link Millions of Addresses with Ten Lines of Code in Ten Minutes

In-Database Machine Learning Illustrated

June 1, 2017

I have just received the excellent news that Apache MADlib, a big data machine learning library for which I was a committer until recently, has graduated to become a top-level Apache project. The basic idea behind MADlib is actually quite interesting and deserves to be more widely known. Massively Parallel Processing (MPP) databases like Greenplum have … More In-Database Machine Learning Illustrated

Large-scale Subscriber Preference Modelling for Telcos – Part 2

November 5, 2016

In Part 1 of this blog article, we looked at the problem of tokenising a URL as an intermediate step towards learning user preference models from browsing histories. In Part 2, we next look at the problem of learning a URL classifier model from the preprocessed Shalla dataset using Support Vector Machines. A standard way … More Large-scale Subscriber Preference Modelling for Telcos – Part 2

Large-scale Subscriber Preference Modelling for Telcos – Part 1

October 30, 2016

An important way telcos can increase revenue is to improve, within the constraints of privacy laws, provision of personalised services for subscribers. To achieve that, they need to be able to build good subscriber preference models. These can take a number of forms, depending on the specific business context and the exact data available. In this … More Large-scale Subscriber Preference Modelling for Telcos – Part 1

PL/Fortran and PL/C++ on PostgreSQL and Greenplum

December 18, 2015

Most modern big data platforms support parallel execution of (non-native) code written in languages like Python, Perl, R, and Java. On Greenplum and HAWQ, two massively parallel relational database systems, these facilities come in the form of PL/Python, PL/Perl, PL/R, and PL/Java, which are inherited from PostgreSQL. These programming facilities are useful for a range … More PL/Fortran and PL/C++ on PostgreSQL and Greenplum