The Perfect Data Storm, Fueled by Big Data & Hybrid Cloud, Looms for 2014

Featured article by Yves de Montcheuil, Vice President of Marketing, Talend

A Perfect Storm describes an event where a rare combination of circumstances will aggravate or amplify a situation drastically. The second half of 2013 has seen more and more big data projects move from sandbox/proof of concept environments to production. With regards to cloud computing, we are now way past the initial hype and entering a new era of hybrid deployments, where the cloud transparently extends the data center.

Several combined developments, reaching maturity in the same timeframe, are circumstances that will be fueling the Perfect Data Storm of 2014.  Here are five predictions that demonstrate how this Perfect Data Storm is brewing.

Big data and the cloud

Using cloud platforms to deploy Hadoop clusters is already a no-brainer for most organizations. However, the primary contribution of cloud computing to big data will soon reside in the cloud being a source (or rather, many sources) of big data. The “public” cloud is full of untapped sources of data: social data, open data, syndicated data and more.

Prediction: In 2014, as big data projects emerge from their sandboxes, they will gather steam from the cloud, beyond deployment elasticity, through the new sources of data it represents.

The big data computing platform

In its “1.0” incarnation, Hadoop is essentially a single-task platform, only able to run one process at a time. With version 2.0, and especially YARN, Hadoop becomes much more a computing platform, able to run all kinds of concurrent workloads, some of them batch with “longer” response times, and some of them interactive/real-time. YARN also extends the ability of Hadoop from just MapReduce to other processing frameworks. Use cases are already plentiful, which will boost the adoption of Hadoop 2.0 and turn it into the “next” enterprise computing platform.

Prediction: 2014 will see Hadoop transition from a single-purpose batch data processing environment to a multi-use computing platform running diverse applications, mixed workloads, and driving more real-time and operational uses of big data.

The hybridization of cloud

Adoption of cloud computing by enterprises is now enacted. From Infrastructure as a Service (IaaS) to Platform as a Service (PaaS), and of course Software as a Service (SaaS), the cloud is pervasive and broadly used to deploy systems, applications and more.  Cloud deployments, however, remain very “siloed”, and systems running in the cloud don’t always work well with on-premises systems. In order to allow a full ubiquitous deployment of applications, regardless of where and how, a real, hybrid integration solution is needed. Integration Platform as a Service (iPaaS) is becoming the enabler that allows cloud and data center to be viewed as one single global IT resource.

Prediction: In 2014, the dichotomy between cloud-based and on-premises will become largely irrelevant. A broader use of virtualization, elastic deployment techniques, and hybrid integration platforms will enable fully hybrid deployment modes.

Pervasive big data

As big data crosses the chasm and transitions into mainstream adoption, most initial uses cases are analytical: understand behavior of customers, perform advanced risk analysis, etc. The goal is to dramatically increase the volume of data that can be stored or processed. With the help of the big data computing platform, insight gathered from big data becomes available not only to reporting applications and human consumers, but also to a broad variety of consuming systems and devices. Now that the technology is becoming available, concrete use cases start to flourish.

Prediction: In 2014, big data will start to be used for operational purposes and will ultimately be consumed by many applications, apps and devices.

The Perfect Data Storm enables the Data Company

The convergence of these developments creates a tremendous opportunity for companies to pivot their business:

Prediction: In 2014, many companies will start to use data to profoundly transform their business, and will either outright monetize their data assets, or at least exploit them to create new business models or exploit untapped segments.

This Perfect Data Storm is coming. Don’t get caught by surprise; be ready to ride it.

Happy New Year!

Yves de Montcheuil is the Vice President of Marketing at Talend, the recognized leader in open source integration. Yves holds a master’s degree in electrical engineering and computer science and has 20 years of experience in software product management, product marketing and corporate marketing. He is also a presenter, author, blogger, social media enthusiast, and can be followed on Twitter: @ydemontcheuil.