The annual pilgrimage to the ESOMAR Conference took place last week in Dublin. I heard that there was much discussion, both on and off the stage, about Big Data and the future of market research. Hopefully, the whole profession will get behind one initiative, instead of each individually trying to “solve world peace” on their own!
This week sees the second Swiss BI-Day taking place in Geneva and there will no doubt be similar discussions about Big Data and the future of Business Intelligence.
It appears that Big Data is not just a buzzword or a commodity that has been likened to oil; it has become the centre of a power struggle between different industries. Many professionals seem to be vying for the right to call themselves “THE Big Data experts”.
This got me thinking about the future of data analysis in general and the business usage of Big Data more specifically. There seems to be no stopping the inflow of information into organisations these days, whether gathered through market research, which is proportionally becoming smaller by the day, or from the smartphones, wearables and RFID chips, that get added to every conceivable article, more generally referred to as the IoT (Internet of Things). Who will, and how are we to better manage it all? That is the question that needs answering – soon! (>>Tweet this<<)
Data Science Central published an interesting article earlier this year called “The Awesome Ways Big Data Is Used Today To Change Our World”. Already being a few months old probably makes it a little out-of-date, in this fast changing world we live in, but I think it still makes fascinating reading. It summarises ten ways that data is being used:
- Understanding and Targeting Customers
- Understanding and Optimizing Business Processes
- Personal Quantification and Performance Optimization
- Improving Healthcare and Public Health
- Improving Sports Performance
- Improving Science and Research
- Optimizing Machine and Device Performance
- Improving Security and Law Enforcement
- Improving and Optimizing Cities and Countries
- Financial Trading
Many of these are not new in terms of data usage nor business analysis. What is new, is that the data analysis is mostly becoming automated and in real-time. In addition, the first and second items, which were largely the domains of market research and business intelligence, are now moving more into the hands of IT and the data scientists. Is this a good or bad thing?
Another article posted on Data Informed a few months after the above one, talks about “The 5 Scariest Ways Big Data is Used Today“ and succinctly summarises some of the dynamic uses of data today. The author of both pieces, Bernard Marr, wrote that “This isn’t all the stuff of science fiction or futurism. Because the technology for big data is advancing so rapidly, rules, regulations, and best practices can’t keep up.” He gives five examples of where data analysis raises certain ethical questions:
- Predictive policing. In February 2014, the Chicago Police Department sent uniformed officers to make “custom notification” visits to individuals whom they had identified, using a computer generated list, as likely to commit a crime in the future. Just one step towards the “Minority Report”?
- Hiring algorithms. Companies are using computerized learning systems to filter and hire job applicants. For example, some of these algorithms have found that, statistically, people with shorter commutes are more likely to stay in a job longer, so the application asks, “How long is your commute?” Statistically, these considerations may be accurate, but are they fair?
- Marketers target vulnerable individuals. Data brokers have begun selling reports that specifically highlight and target financially vulnerable individuals. For example, a data broker might provide a report on retirees with little or no savings to a company providing reverse mortgages, high-cost loans, or other financially risky products. Would we want our own families targeted in this way?
- Driving analysis devices may put you in the wrong insurance category. Since 2011, car insurance companies like Progressive and Axa, have offered a small device you can install in your car to analyze your driving habits and hopefully get you a better rate. But some of the criteria for these lower rates are inherently discriminatory. For example, insurance companies like drivers who stay off the roads late at night and don’t spend much time in their cars, but poorer people are more likely to work the late shift and to have longer commutes to work — both of which would be strikes against them when it comes to calculating their auto insurance rates.
- Walmart and Target determine your life insurance rates. OK, not directly, but Deloitte has developed an algorithm, based on “non-traditional third-party sources” that can predict your life expectancy from your buying habits. They claim that they can accurately predict if people have any one of 17 diseases, including diabetes, tobacco-related cancer, cardiovascular disease, and depression, by analyzing their buying habits.
Marr starts this article by very briefly discussing privacy and inherent biases in data. I think these issues are far more urgent than deciding whether it is market research, business intelligence or data scientists that are in charge of the actual data analysis. Perhaps we all need to work together so that the “Human” side of data is not forgotten? After all, most data comes from people, is understood – if no longer strictly analysed – by people, for the benefit of people, to help change people’s behaviour. What do you think? Join the conversation and let your voice be heard. (I’ll be presenting this very topic at the Swiss BI-Day this coming Tuesday, so I do hope that you will pop by and listen)
This post includes concepts and images from Denyse’s book Winning Customer Centricity. You can buy it in Hardback, Paperback or EBook format in the members area, where you will also find downloadable templates and usually a discount code too.
The book is also available on Amazon, Barnes and Noble, iBook and in all good bookstores. If you prefer an Audiobook version, or even integrated with Kindle using Amazon’s new Whispersync service, it’s coming soon!