nIn the Nov. 3, 2016 HBR there was a wonderful article titled, “You Don’t Need Big Data — You Need the Right Data.” The normal cheerleader rah-rah stance of the HBR was somewhat dulled; acknowledging that data is not always the answer.
Anytime a story runs regarding “how much data exists” my attention is piqued. For example, “By the year 2020 about 1.7 megabytes of new information will be produced every second for every human being and we will be dealing with 40 Zettabytes of data” (Korolit, 2015). I am a little unclear on the math here, as this rate of production based on an estimated population of 7 billion people = 1.2 Petabytes/Second or 72 Petabytes/Minute. Nonetheless this is a lot of data.
Mega –> Giga –> Tera –> Peta –> Exa –> Zetta –> Yotta
This is mind numbing. How much could possibly be learned from this much data? Ok, there is probably a lot we can learn from this, but that is entirely dependent on the ability of computers to analyze the data.
Until computers become self aware (at which point humans will cease to matter) there is a limit to how much valuable information can be extracted from Zettabytes worth of data. Computers are good at performing the same task over and over. This is why self-driving cars will eventually be far superior to any human driver. Self-driving technology will be able to learn from all the data collected by all the other self-driving cars and be better at anticipating actions than any human.
How much value can we get from all this data?
Industries are always looking for the next competitive advantage, and many seem to view big data as just the ticket. This is short sighted. Society continually puts faith in the ability of technology to answer problems. While this happens on occasion it is not the rule. Additionally, 40 Zettabytes of data from who? Will everyone have Internet access? This data will not reflect the actions of those that have no access to the Internet or more importantly the Internet of Things, which is where (I am guessing) all this data will come from.