This week I heard a brilliant lecture by Kenneth Cukier, the digital editor at the Economist, talking about Big Data. He was giving the Anthony Howitt Lecture at CIMA (The Chartered Institute of Management Accountants). The title was ‘Making Decisions Amongst The Data Deluge’, and Cukier painted a picture of a new landscape where every feature of our lives should be treated as a platform for the collection and analysis of data. Help, how Big Brotherly that sounds! Or, alternatively you could say it’s what market researchers have always been doing?!


As an ex market researcher, I was interested in exactly how big data has really changed the way we make decisions. Cukier describes how it is the sheer quantity of data and new processing powers that have been the game changes, for instance, he estimates that the world-wide quantity of digital data doubles every three years. One important area of change in the work place is where machine-learning algorithms can now outperform human expertise. In the past the IQ of a robot was limited by the intelligence of the programmer; but with advancements in AI (Artificial Intelligence) this is no longer the case. Now computers learn for themselves, and they are often better learners than we are. As a consequence, Cukier suggested that it was time to invite machine-educated computers to the board table!


Another example of how big data is changing decision-making is in the area of consumer insight. Apparently, big data analysts are increasingly revealing surprising correlations of consumer behaviour, and these totally unexpected findings are subsequently being used to drive marketing initiatives. For instance, data in the car market recently revealed that if you own an orange car you are less likely to have an accident; and as you can imagine I left the evening wondering why I hadn’t bought an Orange car before!