Profile Similarity

Observing the radar charts of data activities for the different roles, we can already form a mental image of how similar these roles are in reality. However, to acquire a more holistic view, let’s plot everything on a 2-D “map”. We currently have 7 dimensions (the 7 data activities), so we will use PCA to reduce them down to 2. PCA’s top 2 components explain 91.7% of the variance observed in the data, so we have managed to retain quite a lot of information:

• Data Scientists and Business Analysts are the two most dissimilar professions.

• The Data Scientist profession is actually much more similar to the Data Engineer than it is to the Data Analyst.

• Data Analysts, Business Analysts and Statisticians form a little cluster, meaning that their jobs, in terms of data-related activities, have many similarities.

• Data Scientists and Research Scientists seem to have the most “unique” activity patterns, meaning that their individual professions are quite different to all the rest.

Salary Comparison

It was previously mentioned that, on average, a Data Scientist is the highest paid data profession. A more comprehensive comparison across all role is presented below.

Of course, not knowing the exact internal distributions of seniority level and experience within each role, we have to take the below with a pinch of salt, but it still makes for an interesting high-level comparison.

As salary responses are given as a range, each individual’s salary is assumed to be the average of that range.

Cases where the response to the annual salary question was “> $500,000” have been omitted, as they could be anything from 500K to a couple of millions.

• Statistician – Average salary: 111K $ – Median salary: 95K $
• Research Scientist – Average salary: 117K $ – Median salary: 112K $
• Data Scientist – Average salary: 137K $ – Median salary: 137K $
• Data Engineer – Average salary: 125K $ – Median salary: 112K $
• Data Analyst – Average salary: 86K $ – Median salary: 85K $
• Business Analyst – Average salary: 86K $ – Median salary: 85K $

• Data Scientists are the highest paid professionals, followed by Data Engineers and Research Scientists.

Thoughts / Discussion

Data science is a team sport. We work together, we learn from each other’s blogs and Kaggle kernels and we attend each other’s talks in meetups and conferences. I strongly believe that by raising awareness of what other data professionals do, we can collaborate better, as well as build more efficient and comprehensive data teams.

I hope you enjoyed reading through this analysis as much as I enjoyed building it. I also hope that you learnt something you didn’t know or reconsidered about something you though you knew. I know I certainly did.

Some thoughts:

• Linear/Logistic regression and tree-based methods are the most popular choices of algorithms across all professions. Is this due to their ease of use and interpretability or have we done something wrong when it comes to teaching and making accessible more advanced algorithms?

• It seems that many Data Scientists spend time building data infrastructure and many Data Engineers spend time doing data analysis. Is this the best use of their time? Couldn’t this lead to employees being unhappy? How do we raise awareness of this issue?

• Even though many professionals across different roles are practicing machine learning, the fields of computer vision and NLP seem to be uncharted territory for many, even though they are becoming increasingly more important in this age of big data. How do we close this gap?

• Data visualisation was the dominant data activity across all roles and for a good reason. It enables us to understand our data, tell stories, discover insights and engage audiences. So, next time, before you fit that fancy neural network, make sure you visualise your dataset first 🙂