Editor’s Note: This was originally published on 11/9/11 and felt it was worth republishing as many of the fundamental drivers discussed for this industry and company remain intact.
According to EMC (NYSE:EMC) President Pat Gelsinger, Big Data is a $70 billion industry and growing at a rapid rate of 15% to 20% a year. In our view, this helps put to rest the notion that ‘Big Data’ is just a fanciful marketing term.  As IDC puts it, “Big Data developments will be perhaps the most critical new marketplace for storage solutions providers in the coming decade. Providing a strong portfolio of complete Big Data solutions – hardware, software, and implementation services – will be a high priority to succeed.” 
These are indeed exciting times for storage players like EMC, NetApp (NASDAQ:NTAP), HP (NYSE:HPQ), IBM (NYSE:IBM), and Dell (NASDAQ:DELL) among others that now face the challenge of securing their position in an industry undergoing rapid changes. We see outsized returns for the companies well positioned for this trend like EMC.
The cloud enables Big Data
In reality, the concept of Big Data has really kicked in with the rise of cloud-based service providers like iTunes, Netflix, Youtube, Facebook etc. which are primarily in the business of gathering, organizing, and providing access to large quantities of digital content. In addition, cloud-based service providers like Salesforce.com, Amazon web services and others that have focused on delivering IT infrastructure and applications “as a service”.
Over the past several years, these companies have undertaken massive storage build outs as they have expanded their service offerings, entered new markets, and extended their geographic reach.
In parallel to the expansion of the public cloud, many organizations have started to deploy their own private clouds for internal applications and content (archival) storage. Some of these private cloud deployments (e.g., government and research sites) are comparable in scope and complexity to public cloud environments while others are more limited.
Public and private cloud storage demand drives growth
The spending on storage by public cloud service providers is expected to grow at an annual rate of 24% from 2010 to 2015 while enterprise spending on storage for the private cloud will experience annual growth of near 30%.
Though the ‘Big Data’ phenomenon seems to have gained popularity only after the launch of cloud services, in reality, some aspects of this emerging trend are not new.
The data volumes have grown exponentially for several years now and, in fact, a major chunk of the ‘Big Data’ currently housed in massive data centers have nothing to do with cloud services. The overall global spending by public and private cloud service providers on storage hardware, software, and professional services will only be $22.6 billion by 2015 – much less than the $70 billion or so of the total expected size of the ‘Big Data’ industry cited by EMC. 
Storage software and services are the winners
While there is no doubt that ‘Big Data’ is going to be a boon for storage hardware providers due to the sheer growth in volumes expected, the opportunities will be even greater in the storage software and services segments. The real value from ‘Big Data’ is expected to be in the form of high-end analytics, predominantly using data mining, statistics, optimization and forecasting type of capabilities to proactively turn this data into intelligence to drive business benefits and better decision making capabilities.
Given the vastly different nature of ‘Big Data’ from the traditional structured data, the tools required will also have to be different or rather entirely new. 
This becomes even more important as storage software and services are comparatively more valuable as a business than storage hardware as demonstrated in our EMC model above. This is why EMC has a huge appetite for ‘Big Data’ technology and why HP is shelling out $10 billion for a firm like Autonomy whose most important asset seems to be a head start in ‘Big Data’ analytics.Notes:
- EMC has huge appetite for Big Data technology, searchstorage.com [↩]
- Demand from Public Cloud Service Providers and Private Cloud Adopters Will Drive Strong Growth for Full Range of Storage Solutions, IDC [↩] [↩]
- IDC Expects “Big Data” to Drive the Next Wave of Investments in Business Analytics for CIOs in Asia/Pacific, IDC [↩]