Big Data is changing our world. It will completely transform the way we do business, run countries and cities, find love, cure cancer, conduct science and most other things. This is why there is such a massive hype around this buzzword. At the same time there is still little real understanding of what big data really is and why it will change the world. In my other post on the topic entitled ‘Big Data: The Mega-Trend That Will Impact All Our Lives’ I outline how the frightening ‘datafication’ of our lives provides access to more data than ever before. We are now tracking and storing data on everything, which is why we have so much data and why the amount of data is growing at a staggering rate each day. So, yes – we have large volumes of data, big data if you want. But the real hype is NOT about the large volumes of data.
The real hype around ‘big data’ is about what we can now do with this data. It is not the amount of data that is making the difference but our ability to analyze vast and complex data sets beyond anything we could ever do before. I don’t want to go into too much technical detail here, but cloud computing combined with improved network speed as well as innovative techniques to analyze data have resulted in a new ability to turn vast amounts of complex data into value. What’s more, the analysis can now be performed without the need to purchase or build large supercomputers. This means that any business, government body, or indeed anyone can now use big data to improve their decision-making.
Especially powerful is our ability to analyze so called ‘unstructured data’. This is the data we can’t easily store and index in traditional databases such as email conversations, social media posts, video content, photos, voice recordings, sounds, etc. Combining this ‘messy and complex’ data with other more traditional data is where a lot of the value lies. Many are starting to use big data analytics to complement their traditional data analysis in order to get richer and improved insights and smarter decisions. Let’s look at some practical examples of how big data analytics is helping to make our world smarter.
There are so many examples of how businesses are now using big data to become smarter. Take Wal-Mart, who is now able to take data from past buying patterns, their internal stock information, mobile phone location data, social media as well as external weather information and analyze all of this in seconds so it can send someone a voucher for a BBQ cleaner to their smart phone – but only if that person owns a barbeque, the weather is nice and he or she is currently within a 3 miles radius of a Wal-Mart store that has the BBQ cleaner in stock.
Another client of mine, a leading telecom company, has developed big data analytics models to predict customer satisfaction and potential customer churn. Based on phone and text patterns as well as social media analytics the company was able to classify customers into different categories. The analytics showed that people in one specific customer category were much more likely to cancel their contract and move to a competitor. This is extremely useful information that now helps the telecom company closely monitor the satisfaction levels of these clients and prioritize preventative actions.
Here is another example from the world of sport where big data analytics is increasingly used to improve the performance of athletes. The Olympic cycling team in the UK uses bikes that are fitted with sensors on their pedals and collect data on how much acceleration every push on the pedal generates. This allows the team to analyze the performance of every cyclist in every race and every single training session. In addition, the team has started to integrate data from wearable devices (like smart watches) the athletes wear on their wrist. These devices collect data on calorie intake, sleep quality, air quality, exercise levels, etc. The latest innovation now is to integrate analysis of social media posts to better understand the emotional states of athletes and how this might impact track performance.
Big data analytics are currently completely transforming healthcare. One example is a hospital unit that looks after premature and sick babies. The unit is now applying real time analytics based on a recording of every breath and every heartbeat of all babies in their unit. It then analyses the data to identify patterns. Based on the analysis the system can now predict infections 24hrs before the baby shows any visible symptoms. This allows early intervention and treatment that is so vital in fragile babies.
Love is an important element of human happiness and I guess we all want to find our soul mate. But how do we find the right one? Even here big data analytics can help. Take dating site eHarmony. Its founder studied thousands of married couples and based on the findings created a predictive analytics model that takes into account twenty-nine different variables relating to different personality traits, behaviors and social skills. Each person who signs up for the site has to complete a comprehensive profile questionnaire, which will then provide the data for the analytics model to find you a match. This way eHarmony is able to match you with someone that might not fall into your usual dating pattern but where the data suggests a good match. Other match-making sites use different analytical models. Take Perfectmatch.com as another example, their analytics model looks for ‘complementary’ personality traits. Many of the online dating sites are now looking at integrating data from social media networks into their models.
Many cities are now using big data analytics to e.g. analyze and predict congestion levels on their road and transport networks (among many other things). For example, camera data, traffic updates, whether information, train and bus location data, as well as Twitter messages and Facebook updates are all analyzed to get a realistic real time understanding of traffic levels. This can then be used to e.g. re-route busses and trains, increase or decrease the frequency of public transport on specific routes, re-route and optimize traffic flows, etc.
There are so many other examples I could list. Basically, all parts of our lives will soon be affected by the analysis of big data.
As always, please let me know your thoughts on the topic. Do you find it frightening or exciting? Do you see never-ending opportunities or are you worried about the ‘Big Brother’ effect and the exploitation of personal data?