IT Briefcase Exclusive Interview: Best Practices for Large-Scale Database Management

Analyst firms such as Gartner suggest that there is a massive shift from legacy proprietary database systems to open source. Alan Paradise, Director of Database Administration for Wiland, has deep experience working with large proprietary vendors such as Oracle and Microsoft, as well as fast growing open source databases such as MariaDB. In this article, we ask Alan about Wiland’s predictive analytics and audience modeling, modern challenges of managing large relational databases, and how open source fits into Wiland’s strategy.

• Q: Can you define predictive analytics and audience modeling?

A. Anthropologists believe that our hunter-gatherer ancestors survived not just because of speed and strength. Rather they survived because they studied their prey’s behavioral patterns.  They learned to observe the animals’ activities, and scoured the landscape for signs revealing the presence and movement of game. In other words, by observing past behavior, they learned to make predictions about future behavior.

Successful retail businesses today share some of the same traits. Retailers are eager to understand consumer behavior so that they can win future sales and get a jump on their competition.

This is where predictive analytics and audience modeling come into play.

Predictive analytics uses many techniques from data mining, statistical analysis, modeling, machine learning, and artificial intelligence to analyze consumer behavior data so that we can make accurate predictions about future behavior.  And audience modeling helps consumer-facing organizations identify and predict trends among targeted individuals.

• Q: What roles do predictive analytics and audience modeling play in retailing today? What role do they play in other industries?

A. Retailers today are evolving quickly. On one hand, Amazon recently announced it would be adding 100,000 jobs, while on the other hand Sears announced it was closing 150 stores.  As retailers look to thrive in this tumultuous multi-channel environment they have to rely on increasingly advanced tools such as predictive analytics and audience modeling.

While it is easy to measure display advertising simply in terms of clicks and conversions, at Wiland, we take a holistic approach, giving our retail clients a more strategic view of digital marketing opportunities with an emphasis on measurable results. And because our digital solutions leverage our huge, proprietary collection of consumer data, our sophisticated analytics, and predictive audience modeling, we enable our clients to reach not only more customers, but better customers — those most likely to become loyal buyers, donors and subscribers.

Beyond retailing, a number of other industries leverage predictive analytics and audience modeling – industries like publishing, travel and hospitality, automotive, and non-profit fundraising (charitable and political.)  When you have a large continuously updated database containing massive consumer, donor, and subscriber transaction data you can deliver deep insights and actionable intelligence to give businesses a competitive edge in their respective markets.

• Q: How do vendors with large database software products navigate the future?

A. I’ve spent decades working with very large databases. I’ve worked with major proprietary database software vendors including IBM, Oracle and Microsoft.  My experience to this point suggests to me that IT departments have options today they have never had before, in large part due to the rapidly growing availability of excellent open source database software products.

Sure, open source database products are more affordable and they innovate quickly, but there are also the intangibles.  Like, “Who is my account rep and how do they treat me?”  “How quickly do they respond to tech support tickets?”  “How good are they at pushing out bug fixes?”

We work with MariaDB and have been really happy with them.  It is a different kind of partnership – I think because they are a whole different kind of company than Oracle or Microsoft.

I think the large-scale database software market will see a massive shift toward open source in the next decade, and this will help a lot of companies spend less on database software and focus on their core business.

• Q: What are some of the challenges of managing a large database?

A. We have an 8-terabyte database on a single instance of MariaDB. Years ago, database administrators simply didn’t have to work with this kind of scale. But innovations in the marketplace and by open source communities are making the management of this much data much easier.

For example, we see next-generation database proxies such as MariaDB MaxScale as a solution to manage database cluster scalability, availability and query traffic control without throttling application performance.

• Q: What do you suggest for IT departments who are considering their database options?

A. I suggest that IT departments take a close look at what is happening in the open source world. If your relational database has grown to a seemingly unmanageable size over the years, you can look at Hadoop or a NoSQL database – but these solutions may require you to give up the benefits of relational (predictable schema structures, DBA and developer skills, the traditional SQL  language, etc.)  Moving a large relational database to NoSQL or Hadoop would likely require a major application rewrite or developing a lot of new code. We want our developers to create new product features that help us grow revenues.  In our case, it would have cost literally millions of labor dollars to convert our apps to use a new database technology – there was simply no ROI for that.  So we stick to relational and find creative ways to scale it.

• Q: What are the pros and cons of proprietary vs open source databases?

A. At Wiland, we have been quite successful building our predictive analytics tools and audience modeling applications on top of open source databases. We get all the performance, reliability, security, and scalability that we need, and we don’t have the drain of spending millions with proprietary software vendors. Instead, we can direct that investment toward product innovation and revenue growth.

Conclusion:

IT Briefcase thanks Alan for his time and opinion. As analyst firms predict a growing number of companies migrating from legacy and proprietary to modern extensible open source databases, we appreciate Alan’s perspective.

About Alan Paradise: 

Alan Paradise is director, database administration at Wiland, a leading marketing intelligence company serving many of the nation’s top companies and organizations. Paradise has extensive experience with proprietary and open source databases as well as with project management, database administration, systems analysis and design, and technical architecture direction. Paradise is also an adjunct teacher at Webster University and St. Louis University where he teaches classes on project management and Information Systems.