Unspeakable Practices & Unnatural Technical Acts in Data Science/AI
Share
We’ve been through the sexy phase of Data Science and the big hype of Big Data, and now we’re all drinking the AI Kool-Aid…
With all the hype and legitimate promise of AI, many have jumped on the bandwagon and are forging ahead waving the AI flag.
What I’ve observed in the past year, across a number of sectors and domains, that’s been reinforced in conversations with respected peers, is the growth of smoke and mirrors, technical debt, and poor and inefficient practices.
For those of us who’ve been in the tech game for many years, this is nothing new, and was foreseeable, given the exponential growth in the popularity of ML/AI. However, the scale at which new entrants and existing players are cashing in, and rebadging themselves as AI experts, is somewhat unprecedented.
As a result, some organisations are reactive and exhibit short-term thinking, lacking the right leadership to realise true potential. Examples include:
- Blindly relying on vendor products, and ad-hoc technology, without clearly understanding their business problems, or not having senior staff with technical credibility to validate claims and promises, both internal and external.
- Jeopardising effective and efficient pipelines and design, and capability development, to push inadequate or ad-hoc solutions quickly out the door, that don’t scale, resulting in high technical debt.
- Buying into Vanity Data Science/AI (which I’ve previously discussed here & here), resulting in high staff churn/low morale and limited value/unrealised potential.
So, what can be done? Here are some suggestions:
As an employer, ensure you have key staff with technical credibility, to assess claims by vendors, potential candidates, and engagement partners. Also, work to increase the analytics maturity of your organisation, and help develop data literacy at all levels. Don’t forget that productive collaboration between Data Scientists and Software Engineers, and efficient design and processes, is imperative to enable sustainable and scalable ML/AI solutions, especially when transitioning from dev (Data Science) to prod (Software Engineering).
As an employee, be wary of roles that profess to want/need Data Science/AI capabilities, but lack the attitude/maturity/ability to support them.
Furthermore, we need to collectively better learn how to leverage the experience and expertise of the likes of Google and Facebook, to our domains and industries, with their own unique cultures and challenges. They’ve proven how successful integration of Data Scientists, Software Engineers and Business experts can lead to scalable solutions that deliver data-driven actionable insights and results.
I like to think of this as a ‘wicked’ Transfer Learning exercise 
Dr Alex Antic is a trusted and experienced Data & Analytics Leader, Consultant, Advisor, and a highly sought Speaker, Trainer & Advisory Board Member.
He has 18+ years post-PhD experience and knowledge in areas that include Advanced Analytics, Machine Learning, Artificial Intelligence, Mathematics, Statistics and Quantitative Analysis, developed across multiple domains: Federal & State Government, Asset Management, Insurance, Academia, Banking (Investment and Retail) & Consulting.
Alex was recognised in 2021 as one of the Top 5 Analytics Leaders in Australia by IAPA (Institute of Analytics Professionals of Australia). He also holds several senior advisory roles across industry, government, start-ups and academia.
His qualifications include a PhD in Applied Mathematics, First Class Honours in Pure Mathematics, and a double degree in Mathematics & Computer Science.
