AutoML

How Does Automated Machine Learning Work?

by
Jon Reilly
,
June 10, 2021

AutoML, short for automated machine learning, is a method of machine learning in which a computer builds predictive models with minimal human intervention. 

The goal of AutoML is to create a system that can build its own end-to-end machine learning models. AutoML uses an iterative approach in which it seeks to find the best possible algorithm for the task at hand, with varying degrees of automation among the tools used. AutoML enables companies to use artificial intelligence without the typical high barriers to adoption.

AutoML steps

At a high level, AutoML begins with training data - a dataset that contains a combination of attributes alongside a target variable (the thing you’re trying to predict). Algorithms then explore this data and come up with a set of models that best fit the relationship between the attributes and the target. The most accurate model is discovered through tests on held-out or unseen data.

There are generally eight steps in the AutoML process: data ingestion, data preparation, data engineering, model selection, model training, hyperparameter tuning, model deployment, and model updates. Let’s explore these steps in detail and provide insight into why they are important for the process.

Data ingestion

Data ingestion is the first step in the AutoML process, which is where data is read into a workable format and analyzed to ensure that it can be used for the next steps in the AutoML process. 

For example, with Akkio’s AutoML, you can connect datasets from CSV files to Excel sheets to Snowflake data tables. In the data ingestion step, this data is processed and turned into a machine-readable format, such as a Python Pandas DataFrame.

The data ingestion step commonly includes basic data exploration as well, which ensures that the data can be used for machine learning in the first place, such as by verifying that there aren’t too many missing values.

It is important to note that most AutoML software can only be used if there is sufficient labeled data available for the model. Therefore, this step also ensures that there’s enough data available to train a robust model.

Data preparation

The second step in the AutoML process is data preparation, which involves transforming raw data into a clean format that is more appropriate for the model. 

Data preparation, or data preprocessing, can include techniques like deduplication, filling in missing values, scaling, and normalization. Machine learning algorithms can be picky about the data that it takes as input, so this step ensures that the data quality is good enough to be used for modeling.

Data engineering

The third step in the AutoML process is data engineering, which involves choosing how features are extracted and processed, as well as data sampling and shuffling.

Data engineering, or feature engineering, can be done manually, or automatically via machine learning techniques like deep learning, which automatically extracts features from the data and performs feature selection.

Data sampling refers to selecting a subset of the original data for use in training. For example, if an original dataset has 100 entries, then sampling might involve choosing 60 entries from the dataset to use in training. AutoML takes care of this step by randomly selecting certain entries to use in the training data set.

Data shuffling is the process of rearranging pieces of original data into different sequences or configurations before using them in training. This step is sometimes necessary because some algorithms can only be accurately trained using randomly generated data sequences or configurations.

Model selection

The fourth step in the AutoML process is model selection, which involves choosing from a variety of models for model building and training. 

Some models may provide better accuracy on a given dataset or for different tasks, such as binary classification or time series prediction. Choosing a good model can be difficult when you have many to choose from, so it's important to know what information you want to extract from your datasets and what type of model would be best suited for your needs. AutoML tools determine the right model automatically. Some systems use a state-of-the-art technique called neural architecture search for this.

Model training

The fifth step in the AutoML process is model training.

There are many different types of ML models, each with its own set of hyperparameters. Some examples include linear regression models, decision trees, random forest models, neural networks, and deep neural network models. 

Often, several models are trained on subsets of the data, and the most accurate one is selected for further tuning, and ultimately deployment. The final model undergoes a series of validation steps with held-out data. 

Many of the newer no-code platforms create visualizations of model performance making it easy to understand how each AI model works. 

Hyperparameter tuning

For AutoML to work effectively, it must be able to tune hyperparameters, or meta parameters, to maximize performance. This is sometimes called hyperparameter optimization. This means that AutoML systems must be able to generate a series of predictions for different combinations of hyperparameters and then select the best combination, based on its performance.

Some common hyperparameters include initial weights, learning rate, momentum, maximum depth of tree, and others. 

Model deployment

Deploying a trained model after it has been built and tuned can be difficult, especially for large-scale systems that normally require intensive data engineering efforts. 

However, an AutoML system can make building a machine learning pipeline much easier, by leveraging in-built knowledge about how to deploy the model to different systems and environments. With Akkio, there are several effortless options for deployment, including via API, web-app deployment, and deployment to tools like Salesforce, Snowflake, and Zapier.

Model updates

Finally, AutoML systems are also capable of updating models over time as new data becomes available.

This ensures that models are always up-to-date with new information, which is especially important in dynamic business environments.

Benefits of AutoML

One benefit of AutoML is that it removes the need for experts and data scientists to do the time-consuming work of training models and assessing them manually. This can save organizations huge amounts of time and money, as large technical teams of data scientists and software developers would traditionally be needed for this process. In turn, this lets businesses focus on other tasks, such as using the model to optimize their processes and metrics.

To give a more specific example, imagine a company that sells and ships products. This company has a large dataset of customer orders, and wants to use machine learning to predict which customers are likely to order again in the future. Manually training a model to do this would be extremely time-consuming, but with AutoML the whole process can be automated.

Another major benefit of AutoML is that it can help businesses to avoid the costly mistakes that can be made when humans are involved in the modeling process. For instance, if a data scientist develops a model that is overfitted to the training data, it will perform poorly on new data and will be of little use to the company. However, AutoML can help to avoid this problem by automatically selecting and training models that are less likely to overfit, using methods like regularization and dropout, and by providing tools for model sharing and interpretability.

Moreover, AutoML can help businesses to keep up with the latest advances in machine learning, as new methods and algorithms are constantly being developed. Rather than relying on data scientists to stay up-to-date with the latest research, businesses can use AutoML to automatically select and implement the most appropriate methods for their data.

In short, AutoML is a powerful tool that can save businesses a lot of time, money, and effort. It is able to automatically train high-quality models that are less likely to overfit, and can keep business processes up-to-date with the latest advances in machine learning. As such, it is an essential tool for any business that wants to make use of machine learning.

The AI of today is not just voice assistants and translation apps. It is transforming industries from sales and marketing to finance by identifying patterns that were previously unknown or invisible. This is an exciting time for businesses because they’re seeing the impact AI can have on their work more immediately and with far less effort.

AutoML tools

Today, there are several AutoML tools on the market used for various purposes. RunwayML, for instance, is an AutoML toolkit used by creatives for tasks like cutting objects out of videos, creating synthetic images, and more.

Akkio’s AutoML is a no-code tool for non-technical professionals to quickly build and deploy AI for tasks like churn reduction, attrition prediction, fraud detection, and sales funnel optimization.

Most AutoML tools, like Google Cloud AutoML or similar offerings from AWS Sagemaker and Microsoft Azure, are focused on helping technical experts speed up their workflows. These AutoML platforms offer technical solutions to companies, which can be extremely difficult to use for business professionals but can greatly aid in the workflows of AI engineers.

History of AutoML

The history of AutoML is relatively short. It originated in the 1990s as a set of open source tools, like Weka, which helped research scientists and engineers automate tedious tasks. 

These early efforts made machine learning more accessible and accelerated the development of more sophisticated techniques. In the early 2000s, commercial software companies began to emerge that offered proprietary solutions for automated machine learning.

In 2016, Google released their AutoML platform, a technical toolset for developing machine learning models.

In 2020, Akkio released its no-code AutoML platform, the first non-technical AI tool allowing anyone to build and deploy models in minutes.

Future of AutoML

In the future, AutoML will have pivoted mainly towards no-code AI, which makes AI truly effortless.

The advantages of no-code AI over traditional AutoML are numerous. First, it eliminates the high barrier to entry created by the need to know programming languages like Python or make difficult decisions about training hardware (CPU or GPU). No-code AI also eliminates the need for a team of data scientists with extensive data science skills. Perhaps even more importantly, it allows people who don’t have extensive financial resources to build machine learning models and make use of them.

This is all possible because no-code AI platforms make use of drag-and-drop interfaces that allow people without coding skills to upload their data and immediately start training machine learning models. The platforms take care of all the complex calculations and provide users with all the tools they need for quick deployment.

It isn't just that most people can't code; it's that businesses are stretched thin and can't afford to hire data scientists or train their employees in coding languages. Even with technical talent on staff, it can be difficult to find the time to implement and experiment with different AutoML tools. This is where no-code AI shines, as the tedious and time-consuming work is done by the platform.

AutoML

What We Learned From AutoML 2021: Five Key Takeaways From the Fifth International Workshop on Automation in Machine Learning

AutoML

What is Automated Machine Learning (AutoML) and How Does It Work? 

SIGN up

Grow Faster with No-Code ML

Now everyone can leverage the power of AI to grow their business.