Big Data: A Revolution That Will Transform How We Live, Work and Think by Mayer-Schonberger & Cukier has generated a lot of buzz. Is this something you should care about, or even be worried about? Let’s start with some background.
Here are some of the more interesting Big Data findings presented in this book:
– Google can now do a better job of predicting a flu epidemic than the CDC.
– A traffic analyzer program was able to predict a sharp spike in unemployment prior to the government reporting it based on lower volumes of traffic.
– Happiness is correlated with income up to a certain point, and then additional income does not create additional happiness.
– Apple received a patent in 2009 for their earbuds specifically for collecting data on heart rate & body temperature.
Perhaps these are interesting, but how can this help your business? Consider these:
– UPS was able to reduce its vehicle maintenance cost by millions by being able to delay certain preventative maintenance based on analyzing extensive amounts of vehicle failure data.
– Target knows when its female customers are pregnant; and they are pretty good at pinpointing the due date. They generate business by targeting the right coupons throughout the phases of the pregnancy.
– Big Data is based on analyzing (virtually) all the data. Small Data was based on sampling much smaller data sets.
– Big Data is concerned with answering the what (correlation) but not the why (causality). This is a very significant point. The authors argue that knowing the what is for the most part all that you really need to know. Sometimes knowing the why is critically important – therefore Big Data is not always the right approach.
– Small Data involves having a theory before you started to analyze the data. Big Data lets “the data speak”. This is among the most important points. Algorithms will run on the data and reveal answers to questions that you didn’t think to ask. As Duncan Watts wrote, everything is obvious once you know the answer.
So should you care about this? Absolutely! Consider Google vs. Microsoft. Microsoft has the best spell checker in the world, right? In fact, Google does. With the most popular word processor on the planet for many years, you would expect that Microsoft was able to develop the best spell check algorithms. But they did not base it on Big Data. Google did. They collected every misspelled word that users entered. They applied Big Data processing on this data. Now, Google has the best spell checker on the planet.
What can you do? Think about what data you have – good and bad. Assemble your key staff and brainstorm on business innovations that you might be able to make based on your data. You might be able to dramatically improve your service levels. You might find an opportunity to launch a new product or service that will solve a recurring need for your clients. Or, you may be sitting on a gold mine of data and can use it to enter a new industry altogether. It’s not easy. The analysis requires specialized expertise. But, this revolution is here to stay. You should start to think of your data as a strategic part of your business.