At a very high level, Machine Learning is a branch in Artificial Intelligence enabling machines to learn from data to do something without specifically be programmed to do that. Generally, it is about finding patterns in data to:
- Make a prediction based on the situation of specific cases
- Find an abnormality
- Classify a case
Machine learning has applications in many industries such as financial services, healthcare, retail, transportation, and so on. For instance, in financial services, it can be used to identify a client with high risk, spot a fraudulent transaction. Or in retail predict the required stock, recommend products for shopping, etc.
Machine learning has two distinct categories; supervised, and unsupervised learning in which each will solve different types of problems.
How does Machine Learning work?
At a very high level, there is a two-step process in Machine Learning or ML:
At the first stage, at the Learning or Training phase, a human uses a set of tools and provides a set of relevant data for a specific scenario (i.e. price prediction, customer classification, etc.) to a Machine Learning builder application. Then the application uses that data and outputs a Trained Machine Learning model.
Once the Machine Learning application built the trained model, then we can deploy it to an environment and start using it. And what that means is we can pass some situational information (i.e. a current customer purchasing basket information) and the Machine Learning Trained model will tell the system a scoring value and the system decides if it needs to show a customer any relevant information (e.g. if the system should offer the customer with some recommendations on other products for purchase).
Machine Learning Challenges
There are many challenges in both stages of Machine Learning projects. At the Learning phase, it requires a data scientists, a programmer, or a person with statistical analysis skill to:
- Prepare the dataset and selects a sample of data points for training
- Select an algorithm based on current experiences with multiple algorithms
- Configure the selected algorithm
- Create the Trained Machine Learning model (train the machine with data)
- Measures the quality of the trained model
If the person is happy with the quality of the trained model, then the trained model is ready to be deployed to an environment.
The challenge is this training phase requires deep technical knowledge of the data and algorithms. It is heavily dependent on the experience of the person doing it and his past exposure to multiple algorithms.
Therefore, although the ML model is solving a very specific domain problem (i.e. product recommendation, customer risk classification, etc.) the domain expert of the field (e.g. a policy officer) cannot build the ML model and start experimenting with it.
This is a big gap that should be filled with a technical person (i.e. data scientists, programmers, statistical analysts, etc.)
And then, the next step of deploying and using models for real cases is not free of challenges either. Once the ML model is built, it needs to be able to service other processes and applications. This means the need for infrastructure for deployment, securing the potential services, application integration, and so on. This requires IT, software development skills as well as an understanding of processes the ML model will be integrated into.
Machine Learning of the Future
What if professionals and domain experts could actually build the ML models, do experiments with different situations, and deploy them as a service securely and let the processes and applications consume the service as the single source of truth?
What if the machine itself selects the best algorithms for the scenario with the most performing parameters and configuration to build a model in multiple iterations and picks the best-trained machine learning model for us?
Published May 6th, 2021 at 11:04 am