The Digital Company Drives Democratization of Big Data

Featured Blog By Yves de Montcheuil, Vice President of Marketing, Talend

As we have been discussing recently, big data is old news: some organizations have been leveraging their big data to extract tremendous business value, for probably as long as computers have existed. But the operating word here is “some” organizations – not all of them. And these organizations tend to belong to the wealthiest industries – financial, retail or travel to name a few. For these businesses, harvesting and processing vast amounts of diverse data drives them to obtain a competitive advantage that translates into measurable return on investment.

So how does big data get democratized?

The 1990’s and 2000’s have seen the birth of a new phenomenon: the emergence and accelerated growth of the “digital company”: a breed of business entirely based on a digital model. Unlike the more traditional, decades-old or even centuries-old companies that had slowly built their wealth, digital companies have reached incredible market values in less than 10 years.  And unlike the more traditional companies, digital companies have based their model entirely on information technologies.  Where traditional companies would use IT, often very successfully as a tool, even as a competitive differentiator, digital companies are running 100% on IT.

Digital companies are not only using IT as a differentiator, they are modeling IT to their business. Because IT is at the core of everything they do, innovation often means IT innovation. And where a traditional company would design and build new products, digital companies build new IT technologies because they need these innovations.

We have seen very concrete examples of these technologies in the big data domain. The most visible example is Hadoop, for which we can follow a logical path from its inception at Yahoo!, a pure specimen of the digital company, to its becoming mainstream.

• Initially, Hadoop was an internal project at Yahoo! where the technology was being built and used to solve some issues that were very important to the company.
• Because Yahoo! is not a software vendor, they open sourced Hadoop which becomes an Apache project, gathering community contributions – some from user companies, some from vendors or vendors-to-be. However at this stage, users need to be fairly technology-savvy to use Hadoop.
• At a later stage, vendors join the fray and start to offer enterprise-grade distributions of Hadoop, enriched with new utilities: installer, cluster management, monitoring, metadata management, etc. They also offer support, SLAs, references. Hadoop enters a maturity phase. It’s still an early stage technology, but it’s now possible for enterprises to use it outside of a pure experimentation lab.
• The new step is that mainstream begins, with Hadoop being deployed in the enterprise.

Had Yahoo not invented Hadoop, and many other digital companies contributed to make it more powerful and richer, only the wealthiest organizations would still be able to leverage big data. But because digital companies invented Hadoop, big data is now a reality for many businesses.

Yves de Montcheuil is the Vice President of Marketing at Talend, the recognized leader in open source integration. Yves holds a master’s degree in electrical engineering and computer science and has 20 years of experience in software product management, product marketing and corporate marketing. He is also a presenter, author, blogger, social media enthusiast, and can be followed on Twitter: @ydemontcheuil.