Featured article by Lucian Constantin, Independent Technology Author
Demand for data analysts continues to explode, as the IT skills shortage intensifies
For years, database experts struggled to find suitable positions that demanded their unique skillset – software developers seemed to be rapidly moving into their field, often boasting of being proficient with the “Full Stack” of technologies and techniques involved in creating data-driven computer software.
This was largely true between 2003-2008, when Linux / Apache / MySQL / PHP (LAMP) stack developers were more than capable of dealing with the data needs of most small-to-medium sized companies. The role and utility of “Data Analysts” was further eroded by the advent of NoSQL databases such as MongoDB and RavenDB, which seemingly made it easier than ever for IT professionals with non-data-specific backgrounds to accomplish the tasks which had traditionally been performed by Data Analysts.
The Rebirth of the Role of Data Analyst
Suddenly, something changed; data sets begun to balloon in size in an unprecedented way, and development teams found themselves having to concentrate on creating software that could optimize and organize these huge data sets. They didn’t have time to deal with extrapolating and interpreting that data as well.
Data analysts are responsible for using large data sets to create projections, which in turn allow them to produce reports and advisory documents. This requires a deep understanding of how to create and store data in efficient ways, which, when done correctly, allows these talented individuals to identify the efficient and inefficient areas of business performance, suggest possible improvements, and most recently: identify suitable data sets which can be used to train AI-based systems.
Finding the Turning Point
The growth in demand for comprehensive data analytics potentially began around the time that Google launched their Google Analytics product, which allowed anyone with a website to begin tracking vast amounts of data about who was visiting that site on a day-to-day basis.
The advantages of a system like this are difficult to even quantify, especially for companies that were already using Google’s AdWords platform as a major pillar of their online marketing strategy. Overnight, AdWords customers gained the ability to monitor the demographics of their visitors – which country they lived in, how old they were, and whether they were male or female, just for starters.
Although some clever webmasters had known how to track the source of their visitors prior to the launch of Google Analytics, this one single piece of software levelled the playing field at a stroke – now everyone had the ability to view charts showing what keywords were directing traffic to their site, the percentage of “Organic” hits vs those that resulted from AdWords spending, which other websites were sending them significant traffic and how many people were simply typing in the URL directly.
For a site receiving more than a few hundred hits a day, the amount of information created by Google Analytics was simply mind blowing – collating this data and working out how to use it to your advantage required a great deal of time and effort, but the rewards for doing so were obvious. I believe that this is very likely the point where the jobs market for data analysts began to explode.
Which Industries are Experiencing the Highest Demand for Data Analysts?
This sudden increase in data analysis roles caught many companies off guard; this lead to a dramatic increase in demand for companies who focused specifically on Business Intelligence. Today, this is a huge industry and accounts for over 20% of data analytics roles as of 2020. Companies focusing on Business Intelligence will often have specialist teams dedicated to marketing analysis, profit/loss analysis, sales/conversions analysis, and many more.
Some companies still hire their own data analysts, whilst others will outsource the task to these dedicated business intelligence firms. Research indicates that the industries with the highest need for talented data analysts include:
Healthcare
Despite the obvious advantages of electronic prescriptions, distributed patient records, tracking practitioner performance, and analyzing the effect of different treatments on different demographics, the healthcare industry in most countries has only just begun to realize the value of the data they have. Demand in this sector for professional data analysis professionals is enormous, with average US salaries in the region of around $75,000.
Entertainment
The entertainment industry has been expanding and evolving at breathtaking pace over the last 18 months. Online markets such as video streaming, poker and casino sites, and video games are all crying out for data analysts.
Online casino operators are using big data to make decisions regarding which slots and games are most popular with individual demographics and have found the simple action of sorting their content according to individual preferences can have a huge impact on spend – and therefore profit.
Netflix believes that 80% of their current revenue is a direct result of their increased focus on data analytics, which explains why average salaries for these roles are quickly reaching the $100,000 level.
Finance
This is perhaps the most obvious industry with the potential to benefit significantly from careful analysis of large amounts of data. Banks have always needed to store large amounts of data in a safe and secure way, which was extraordinarily difficult in the days before cheap, high-capacity hard drives. As the years have gone by, the problem has continued to increase in size, scale and complexity, despite high-tech Storage Area Network (SAN) server products and high-capacity backup facilities and cloud storage.
Because of the huge size of the datasets in use by financial institutions, Data Analysts are commonly accompanied by Data Scientists and specialist Database Developers, who all have to work together to create working solutions.
Surprisingly, average starting salaries in the US are lower than the previous two examples at around $60,000, although there is usually significant room for progression within each institution. Data Scientists can expect to receive a salary of over $100,000.
Sharing Economy Services
This is an umbrella term which defines business structure rather than a specific industry. Online marketplaces which allow sales from private individuals such as eBay and Amazon fall into this category, along with transportation giants Uber and Lyft, food delivery services such as GrubHub, AirBNB, and even surprise hit Rover, the dog-walking app.
These businesses simply could not function without their use of big data. As with financial, database engineers are critical to these companies in addition to data analysts. Because these companies are entirely dependent on their teams of talented data professionals, they will often headhunt the best talent relentlessly – salaries start at around $70,000, with a median of roughly $90,000 – $100,000.
