Why should business leaders learn to apply Automated Machine Learning?
From data analysis to decision engineering: Why business leaders should learn to apply Automated Machine Learning
The largest companies in the modern world as measured by equity value (market capitalisation) are high tech companies – Microsoft, Apple, Amazon, Alphabet/Google and Facebook. These companies have data as a key asset and have proven that this data is more valuable than oil, industrial capacity or branding. The total value of these companies also broke through the theoretical maximum value of $450b with Microsoft and Apple hovering around $1t. They proved that our view of the maximum size of a company was limited by old world thinking.
Whilst technology plays a key role, many of the most successful players were not the pioneers of the technology upon which they became successful – google was not the first search engine and amazon was not the first electronic marketplace. The differentiating factor was an ability to leverage the technology – to organise for fast growth and to be more successful than existing competitors. The secret to technology success was organisational capability to use the technology, rather than the technology itself. The use of technology lead to the acquisition of data and that was at the heart of the business model. Data and the ability to leverage it for commercial value drove the market capitalisation to record levels.
Through this ability the leaders show that the systematic collection and analysis of data provides a basis for better understanding of client needs, design of new propositions and measurement of the adoption and effectiveness of new products and services. The “platform-based business” and “data driven enterprise” models have proven to be successful, highly scalable and popular models – so powerful that they can create a winner takes all situation. This may also trigger market monopoly problems, privacy concerns and considerable IT security risks – a topic I will not focus on here.
We should take note that whilst Apple and Microsoft have been around for 40+ years, many of the other leaders have a significantly younger heritage – 10 years or less, giving rise to the term “unicorn” – a startup company achieving a billion dollars in market capitalisation. A new model of success was born.
Leadership in a data driven enterprise
The ability to lead a data driven business has become a key determinant for success. Whether your business is a high tech startup, fast growing scaleup or a traditional business based on hard assets, the ability to look for and capture data, gain insight around customer needs and determine how best to meet these needs will greatly increase the chance of success. This created a whole new paradigm for management – where we have relied upon intuition and experience in the past, we can now look to managers who focus on generating insight on customer needs and enabling their organisation to rally around acting upon this. The dominance of the top down command and control structures gave way to more agile, holocratic and fluid organisational forms that enabled people to act more quickly using customer and product data to guide their actions, and result data to determine if these actions were successful. This in real time and directly fed back to the organisation!
A new technique in management has therefore emerged – it is referred to as “decision engineering” which may seem a rather technical term but comes down to a very simple premise – as managers we should place more focus on ensuring that our decisions are better. In a world where competitors are also aware of the data enabled business model – this implies that managers must be better able to engineer decisions than their competitors. That is – understand what decisions they want to make, what data would be required to support and inform the decision and how to action this.
In short – managers who engineer better decisions will be the winners.
How to engineer better decisions?
As a consultant working with many large, medium and startup companies I have seen many problems in achieving this goal – large companies can be swamped with inconsistent, contradicting or useless data. Small companies may not have the resources or infrastructure to acquire the data they want. “Big Data” is not the same as useful data. The key success factor is having relevant and actionable data upon which decisions can be made and assessed.
Following the Big Data hype, we have seen the rise of the discipline of data analytics – more of less over the past 10 years. With the increasing availability of data and understanding of its potential, organisations set up teams to analyse this data. The cost of collecting, communicating, storing and analysing data reduced (cloud, Moore’s law) and the falling cost of infrastructure. Most of the algorithms that we can use to analyse the data such as (linear) regression and classification have been around for 60 years – the difference is that the tools to apply these algorithms have become more widespread and available at low cost or even free. The more recent emergence of unsupervised learning such as cluster analysis or anomaly detection and then neural networks has given rise to new tools which support these advanced algorithms at low cost – often in free libraries.
This explains the widespread use of languages to manipulate data such as R or Python and libraries of software to apply algorithms and generate predictive models such as TensorFlow and Keras. The manipulation of data however remains largely a laborious task which has become the domain of the data analytics teams. They are specialised in using programming languages and libraries of software to piece together the steps in collecting, cleaning, analysing and testing the trustworthiness of models. Indeed, this becomes the focus of time in a data analytics project – we spend the majority of time manipulating the data and a small amount of time actually applying the model and engineering the decisions that managers want to take!
From data analysis to decision engineering!
The trap here is that business managers cannot spend their time mechanically sorting, cleaning and preparing data in the format needed to input this to a tool – they want to focus on the interpretation and meaning of the data and the reliability of the decisions that are based upon this.
I would liken this to the emergence of game computers – they were so simple that a child could literally unbox the product, plug it into the television and start playing a game within minutes. No need to analyse game data, read complex instructions or adapt to the technology – the successful game developers hid the complexity of their technology and made it child’s play! A child could quickly get into a game and compete against other children and use the product with minimal training.
So how to apply this to decision making? A set of new tools is emerging to automate the stages of data analytics which transforms the activity to be focused on business decision making rather than the mechanical steps of data analysis. We focus on the outcome rather than the process! This is called Automated Machine Learning (AML).
The difference with traditional data analytics is highlighted below:
| Stage | Data Analytics Approach | AML Approach |
| Organising the analysis of data | Manually programmed for each model. | No code / low code approach with workflow inherent in tool |
| Applying an algorithm | External software libraries imported and algorithm called by bespoke software | Algorithms clicked as icons on the screen |
| Assessing the outcome | Result display functions called using programmed steps | Results shown in a user friendly manner on screen |
| Managing test and modelling data | Manually split data in test and learning sets | Automatically split data and apply using tool |
| Assessing the outcome | Manually interpret model outcome by viewing results | Automatically assess model outcome on screen |
| Choosing the best algorithm | Specialist choice based on interpretation | Automatic choice based on best performance on test data |
| Deployment of resulting prediction model | Bespoke software steps to deploy model to brouwser or smartphone | Automatic download or model to local browser or smartphone |
| Driven by | Data analytics staff | Business managers |
| Focus of time spent | Analysing the data | Making business decisions |
This brings us back to the beginning of this article – the use of automated machine learning tools will be a key differentiator in the success of business leaders. In my view this is as significant as the shift from the abacus/slide rule to calculators or the shift from calculators to excel spreadsheets. The next shift is from spreadsheets to automated machine learning!
The key to this is learning to use automated tools to focus time on business decision engineering and drive the quest for data, decisions and better business results.
As MIT professors Erik Brynjolfsson and Andrew McAfee stated in “The Business of Artificial Intelligence”: …over the next decade, AI won’t replace managers, but managers who use AI will replace those who don’t.”
