Data Scientist: the most ambiguous role of the 21st century
Data Scientists can really struggle with the ambiguous nature of their role. It's stressed me out in the past - today I'll share how data teams can use ambiguity to their advantage.
In my last post, I drove home the significance of data culture and the adoption of a North Star metric to steer your business. One reader astutely pointed out that my advice resembled the role of a product manager. This sparked a thought - the mind-boggling ambiguity surrounding data roles, particularly in data science.
Ambiguity, a double-edged sword, can either propel your data team to soaring heights of success or send them spiraling into the depths of disarray. When mishandled, it breeds anxiety and confusion, resulting in underperforming, disgruntled teams daydreaming about their next vacation. However, when embraced, ambiguity transforms your data science team into a powerful alliance of superheroes, with each member assuming the roles of data engineers, machine learning engineers, and product managers - joining forces to save the day (or your business).
Grasping this concept is crucial if you're looking to unleash the full potential of your data science team.
Now, consider these two scenarios and ask yourself which is more likely to stress out a data scientist:
A data scientist joins an organization as the sole technical talent, with a remote team of engineers that offer minimal support.
A data scientist joins a well-rounded team, complete with dedicated data engineers, software engineers, machine learning engineers, AI researchers, and product managers, all working toward a common goal.
Surprisingly, it's the second scenario that poses a greater risk. If this rings true for your organization, then this newsletter is tailored for you.
Data Scientists: The Jacks-of-all-Trades
Data scientists are natural-born builders. In the first scenario from the intro, a skilled data scientist will creatively navigate any lack of formal support. They'll craft solutions that, while perhaps makeshift, get the job done and deliver tangible results. This was my experience at Amazon. I didn't have a team of data engineers or machine learning engineers backing me up. Instead, I acquired the necessary skills to patch together a data pipeline and put a model into production. And voila, the machine learning model went live, saving millions annually! I learned a ton, and it remains one of the most enjoyable projects I've ever tackled. (It didn't hurt that our internal AWS budget was virtually limitless, so any cloud-related mishaps and their $$$ consequences weren't a concern.)
Navigating Ambiguity and Technical Difficulty: Don't Get Stuck
1 Low Ambiguity / Low Technical Difficulty
Let's begin with the straightforward quadrant: low ambiguity and simple technical problems. Your data team can breeze through these, delivering value quickly and independently, especially in the early days of your company, team, or project. However, as things mature, staying in this quadrant won't yield the same impact. Data-backed roadmaps collecting dust, weekly reports with no actionable insights, or overengineering your code are all potential pitfalls. The risk of lingering in this quadrant correlates with the level of ambiguity in the environment (it’s comfortable). As a leader, it's your responsibility to ensure that teams don't get stuck in this quadrant. Focus on rewarding impact rather than noise.
2 Low Ambiguity / High Technical Difficulty
Now, let's examine the quadrant with low ambiguity but complex technical challenges. If you're surrounded by data engineers, machine learning engineers, or other specialized technical stakeholders, your data science team may need to take on the "Aligner" role. They'll define the problem, outline high-level steps to tackle it, and estimate potential impact to convince other teams to prioritize the project. Moreover, they'll be responsible for defining and tracking success metrics.
Why not delegate this to a project manager? Because data scientists, as jacks-of-all-trades, speak the same language as other technical teams. They can effectively act as high-level solution designers and assemble the right teams for technical projects. If your data science team is the only technical team, they will adapt, learn, and piece together a solution.
3 High Ambiguity / Low Technical Difficulty
Navigating high ambiguity and low technical difficulty requires a savvy approach. Seasoned team members often have an advantage, as their experience with the company's inner workings and relationships can help them identify levers to move things along. However, newcomers can also excel in this quadrant by seeking guidance from tenured colleagues and learning about the company's product and systems. In fact, newcomers are more prone to offer a fresh perspective on these ambiguous problems, making them valuable resources as well. The key is to balance these two resources well for optimal results.
Thriving in the face of ambiguity not only makes your job more enjoyable, but it also positions you for success since many problems fall into this type of quadrant. By embracing ambiguity and avoiding the trap of seeking only low ambiguity/high technical difficulty projects, you'll be better equipped to tackle a broader range of impactful work.
4 High Ambiguity / High Technical Difficulty
Projects with both high ambiguity and technical difficulty present a significant risk to your team. These "shiny" initiatives may consume considerable resources while offering little in return.
Highly ambiguous/technical projects are a huge risk to your team. They're almost always shiny by nature, and can sink the majority of your team's resources chasing a pipe dream if not handled well. As a leader, it's your responsibility to ensure that the potential impact of a project is worth pursuing before allocating resources.
It's crucial to differentiate between impactful projects and those that are simply shiny. While impactful projects can revolutionize industries and bring about significant positive change, shiny projects may appear exciting but ultimately offer little value.
For low ambiguity/high technical projects, I mentioned that data scientists could serve as "Aligners," driving projects with clear roadmaps and collaborating with technical teams. On the other hand, high ambiguity/high technical projects often lack straightforward roadmaps, making it difficult to maintain focus and ensure everyone is on the same page. Crafting these roadmaps is a challenging task, and even when successfully developed, these projects may still face a lower likelihood of success due to their inherent complexity and uncertainty.
The key to success in this quadrant often lies in identifying unglamorous, hidden opportunities. These projects may not garner much attention initially, but they can provide data scientists with the luxury of time to think, build, and iterate. As the project picks up steam and initial positive results come in, you can start bringing on additional headcount to work on it. With a clear project lead and direction in place, the ambiguity takes care of itself.
The job of a leader is to identify these hidden opportunities and ensure that work starts incrementally. Your role here is to create the opposite of hype; you need to create space. If the project gains momentum, that's great! If not, pivot and learn from the experience. Your greatest asset as a leader is your ability to spot projects with the potential for significant future impact.
Final Thoughts
Ambiguity is the secret sauce that can catapult your data science team to greatness. Embracing it and fostering a culture that thrives under uncertainty will unleash your team's full potential. By navigating the complexities of the data landscape with creativity and adaptability, your data scientists will emerge as the trailblazers of the 21st century. Embrace the art of thriving under ambiguity - it's time to revolutionize your organization's data culture!
Cheers to a brighter future,
Rémi
P.S. I'd love to hear your thoughts and experiences with ambiguity in Data Science or leading data teams! Don't hesitate to hit "reply" and share your stories or insights. I genuinely enjoy talking and reading about these topics.