Data scientists are in high demand as executives seek talented individuals capable of unlocking the hidden value from big data to create big business results. But managing such scarce talent requires a proactive approach over the short, medium and long term.
Big data is enabling companies to gain new insights into areas ranging from customer behaviour to how their businesses function. These data-driven findings can have profound results on the bottom line.
The explosive growth in the amount of available data and an appreciation of its value is going to continue driving very strong demand for data scientists. In fact, their rarity already has some executives describing them as “unicorns”.
British Airways and other major companies recognise that getting the best from their data scientists, however, requires more than just hiring smart people and setting them loose to analyse data.
They believe that data scientists are more effective and bring more value to the business when they work within teams. Innovation has usually been found to occur within team environments where there are multiple skills, rather than because someone working in isolation has a brilliant idea, as often portrayed in TV dramas.
British Airways, for instance, formed its “Know Me” team to develop a more personalised customer approach. Sales people, quantitative experts from the operations research team, data experts from the business intelligence team and some external analytics experts work together to gain deeper insights into customer needs and to roll out new initiatives.
In the short term, the team approach can be taken a step further in an effort to help bridge the data scientist shortage.
The data scientist’s skills – advanced analytics, data integration, software development, creativity, good communications skills and business acumen – often already exist in an organisation. Just not in a single person.
Such people are also likely to be spread over different roles, such as statisticians, bio-chemists, programmers, computer scientists and business analysts. And they’re easier to find and hire than data scientists.
However, over the medium term, recruiting and then retaining data scientists requires an understanding of how they tick.
For many of them, it isn’t only about compensation – though it is important for them to understand what they earn versus their peers. Also, unlike many MBAs, data scientists often don’t harbour ambitions to be the next CEO.
Instead, they are more motivated by recognition and respect from their peers than by the opinion of their supervisors. They are more inspired by cracking difficult problems than by implementing and maintaining the solutions they come up with.
As such, data scientists should be treated as a separate workforce even if they aren’t that numerous within the organisation.
Their career paths also need to be mapped out so that they feel fulfilled. Data scientists need to be kept at the forefront of solving the most difficult problems the business faces.
Companies such as Walmart are not afraid to give their data scientists difficult tasks. They take a pragmatic view and treat them as they do others involved in difficult and future-orientated quantitative tasks. At the same time, they recognise that data scientists work with unique types of data – such as unstructured information flows from social media channels.
Solving the shortage of data scientists over the long term ultimately comes down to the education system. Fortunately, university courses are being created to tackle the issue.
Professor Peter Millican of Hertford College, Oxford, has established a joint degree course in computer science and philosophy. Computer science is about quantitative and analytical skills, while philosophy focuses on flexible thinking, structuring logical arguments and communicating them.
Millican says: “Philosophy habituates students in asking fundamental questions about things that most of us take for granted, and seeking answers in novel ways. These skills are particularly valuable for data scientists, and more broadly in today’s ever-changing world, where young people are likely to have to reinvent themselves several times during their lifetime of employment.”
Others running courses related to data science include the universities of Southampton, Leeds and the Royal Holloway University. In the US, Stanford University has a degree in symbolic systems, which is a mix of computer science, linguistics, maths, cognitive science and philosophy.
However, it will take time for these courses to stem the growing shortage of data scientists. In the meantime, companies can make good use of the talent they already have through team work. Retaining the data scientists they recruit requires designing career paths that meet their need for intellectual challenges and that give them appropriate recognition.
Jeanne G. Harris is a lecturer at Columbia University and Ray Eitel-Porter is managing director for Accenture Analytics in the UK