Mental Models 4 Life

It’s not always easy for a technically competent data scientist to make the transition to a data science leadership role. Here are some interview questions I use to assess whether a candidate has successfully made that transition.

1. Can you tell us a little bit about yourself and why you applied for this role?
2. What is your understanding of our company and how do you think artificial intelligence can help our operations?
3. Can you describe the most successful and most disastrous analytics projects you have personally led, what did you achieve in concrete terms and what did you learn in each case?
4. Can you talk about some of the challenges of managing a team of data scientists?
5. Where do you think an advanced analytics function should fit within an organisation? In business or in IT? What are the trade-offs?
6. Artificial intelligence and machine learning technologies are now pervasive in our daily lives. What do you think the community needs to do to improve fairness, accountability, and transparency in the use of AI algorithms?

Question 1 is an ice-breaker to put the candidate at ease, since almost everyone would have rehearsed an answer to that question.

Question 2 is there to test whether the candidate has done the background reading on your company / organisation, and whether the candidate has a sufficient wide experience across multiple industries to be able to see advanced analytics opportunities in possibly new areas.

For Question 3, we are looking for projects where the candidate has led, and not just participated in. We are also looking for concrete outcomes, e.g. numbers like dollars saved / earned, percentage improvements in business metrics, etc.

For Question 4, we are trying to assess the candidate’s experience in  managing independent-minded data scientists (aka herding leopards), including ways of addressing the needs of highly technical people in a traditional organisation, challenges in recruiting and retaining good data scientists, etc.

For Question 5, the correct answer is of course “it depends on the context!”. What we want to know is whether the candidate has experience with different analytics setups, and their experience working with both business and IT from “the other side of the fence”.

For Question 6, we are looking for the candidate’s awareness of recent trends and discussions in building fairness, accountability and transparency into AI algorithms.

If you can find someone who can answer all 6 questions well, move quickly on them!