Can Companies Keep Up with the Big Data Tidal Wave?
By James D'Arezzo, CEO, Condusiv Technologies
There is no question we are undergoing a sea change in the use of data globally. We have reached a tipping point where nearly infinite amounts of data can be captured and analyzed. Big Data is a tsunami sweeping across organizations everywhere. But the challenge is, how do we manage this unprecedented rise in data to be stored, processed and usefully put to work? According to one study, an estimated 40 zettabytes (43 trillion gigabytes) of data will be created by 2020 (an increase of 300 times the amount of data in circulation in 2005), at a rate of 2.5 quintillion bytes per day.
Collecting data won't be much of a problem. The real issue will be storage, accessibility and performance. There are plenty of big players and start-ups creating solutions that utilize this data for applications such as Artificial Intelligence, the Internet of Things (IoT), and Social Media. These apps demand incredibly powerful computing, high-speed networks and storage response times that are far above today's standards.
One field being heavily impacted by both the IoT and the need for big data analytics is healthcare. More digital tools are being brought into health IT ecosystems for both patients and clinicians to use. More medical images, which take up a tremendous amount of storage space, are also being produced. Human genome sets alone consist of hundreds of gigabytes, and the amount of sequence data is doubling every seven to nine months. Meanwhile, many healthcare institutions cannot afford the cost or lack the space to add enough physical servers to keep up with the demands of big data.
In the business sector, a major across-the-board function-marketing-has gone from being a largely analog and promotional activity to a heavily digitized means of delivering business growth. In the process, marketing has come to rival, and may soon surpass, traditional IT as a center for technology spend. A recent Gartner Group study suggests that marketing leaders now devote a portion of their expense budget to technology equal to 3.24% of total enterprise revenue, just barely behind the current CIO technology spend of 3.4% of revenue.
Social media companies, most of whose business models are centered around the sale and use of data analytics, are both major users of, and investors in, infrastructure designed to keep up with the burgeoning universe of data. It has been estimated that Facebook, for example, spends somewhere above $1.5 billion per month on hosting-related costs.4
To keep up, not just with the overall demands of big data but also with their competitors, organizations will have to get the most out of their IT technology. Part of the solution may be buying new hardware, but there are quicker and more cost-effective things IT system managers can do. One is to implement software that reduces input and output, which can improve performance dramatically. Condusiv has seen users of our software solutions more than double the I/O capability of storage and servers, including SQL servers, in their current configurations.
The point is, the Big Data tidal wave is here and IT pros need a plan to keep their heads above water.
About the Author:
Jim D'Arezzo has had a long and distinguished career in high technology. First serving on the IBM management team that introduced the IBM Personal Computer in the 1980s, he then joined start-up Compaq Computer as an original corporate officer and helped the company grow to over $3 billion as VP Corporate Marketing and later VP International Marketing. Seeing the technology trend toward networking, Jim joined Banyan Systems in the early 1990s as VP Marketing and helped that global networking software leader grow rapidly and eventually go public on NASDAQ. He then moved on to computer-aided design software leader Autodesk as VP Marketing and multiple Division GM for data management, data publishing and geographic information systems. D'Arezzo later served as President and COO for Radiant Logic, Inc., the world leader in virtual directory database solutions. Jim holds a BA from Johns Hopkins University and an MBA from Fordham University.