Do you want to learn about some of the most common model training paradigms that can be used to train machine learning models? Or maybe you want to learn how to choose the right model training paradigm for your situation? Well either way, you are in the right place! In this article, we tell you everything you need to know to choose the right model training paradigm for your machine learning project.
We will start out by talking about what a model training paradigm is and how different model training paradigms differ from each other. After that, we will provide some tips on how to choose the right model training paradigm for your situation. Finally, we will discuss some of the most common model training paradigms that are used in machine learning and provide information on when each paradigm should be used.
What are model training paradigms?
What is a model training paradigm? If you follow machine learning literature then you may have heard terms like supervised learning, unsupervised learning, and reinforcement learning thrown around. Each of these is an example of a model training paradigm. In general, a model training paradigms prescribes what type of task a machine learning model will perform and what constraints the model must operate under. Within each model training paradigm there will be many different types of machine learning models that can achieve the perspired task operating under the prescribed constraints. The specific techniques that are used to achieve the task may vary, but the overall type of task will remain the same.
When you are deciding what type of model you should use for a machine learning project, you should first decide what model training paradigm is appropriate for your project. Once you have decided what model training paradigm you will use, this will narrow down the number of machine learning models that fall within that paradigm. From there you can decide which specific model you will use.
How to choose the right model training paradigms
How do you choose the right model training paradigm to use for your machine learning model? In this section, we will discuss some of the factors that you should consider when deciding what model training paradigm to use for your machine learning model. Here are the primary factors that should be considered.
- What is your objective? The first factor that you should consider when deciding which model training paradigm to use for your project is what your main objective is. Are you looking to create a model that assigns a categorical label or continuous value to each record in your dataset? Are you looking to discover clusters of similar records in your dataset? Are you looking to create a model that can determine which series of steps should be taken to achieve a given goal? These are just a few examples of types of objectives that a machine learning model might aim to achieve. Different model training paradigms aim to achieve different types of objectives, so having information on what your objective is should already narrow you down to a few model training paradigms that are appropriate for your situation.
- What data is available? Once you have determined what your objective is and what paradigms may be appropriate for that objective, the next factor you should consider is what data is available to you. When you are deciding between model training paradigms that are used to achieve a specific type of objective, the main differentiator is often how much data is required and what type of data is required. Some model training paradigms vary from others because they are intended to be used in situations where there is less data available.
Common model training paradigms
What are some of the most common model training paradigms that are used in machine learning? Here are some of the most common learning paradigms that are used when training a machine learning model
- Supervised learning. Supervised learning is by far the most common model training paradigm that is used in machine learning. Supervised learning is used when you want to assign each record with a label, such as a category label or an appropriate numeric value. There are two common subsets of supervised learning models – classification models, which assign a categorical label to each record and regression models, which assign a numeric value to each record. Supervised learning is used when all of the records in your dataset have accurate labels that can be used to train the model.
- Transfer learning. Transfer learning is a paradigm where you take a large base model that has been pre-trained on one task, then adapt that model to be reused on a different task. This is a common paradigm to use when you only have a small amount of data available to train your model, but you want to use a large model like a deep learning model.
- Weakly supervised learning. Weakly supervised learning is a large umbrella term that covers multiple different learning paradigms that can be used when you have data with imperfect labels. Specifically, weakly supervised techniques can be used when your data has incomplete labels (when not all of the data is labeled), inaccurate labels (when the labels are noisy and sometimes incorrect), or inexact labels (when the labels are too coarse). While weakly supervised learning does encompass all of these techniques, it is common for people to use the term weakly supervised learning when they are specifically talking about weakly supervised learning for inaccurate labels.
- Active learning. Semi supervised learning is a subset of weakly supervised learning that can be used when your data has incomplete labels. This simply means that some of your data has labels and other data does not have labels. It is generally the case that there is a small subset of data that has labels that is not quite large enough to train a supervised model on, but there is a large amount of unlabeled data available. What makes active learning methods different from other methods that can be used with incomplete labels is that they require iterative human feedback to be provided throughout the model training process. This feedback generally comes in the form of adding labels to some examples of unlabeled data.
- Semi supervised learning. Semi supervised learning is another subset of weakly supervised learning that can be used when you have incomplete labels. The main difference between semi supervised learning and active learning is that semi supervised learning methods do not require additional human feedback to be provided throughout the model training cycle.
- Self supervised learning. Self supervised learning is used when you have a large body of unlabeled data that you want to use to train a model in a supervised fashion. Self supervised learning methods work by exploring the natural structure and relationships in the dataset to create labels that can be used to train a supervised model. For example, if you have a corpus of unlabeled text data you could create labels for your data by masking one word in each sentence then asking the model to predict the masked word.
- Reinforcement learning. Reinforcement learning is used when you want a model to choose a sequence of actions that should be taken in order to achieve some goal. The model can choose a series of actions to take then depending on whether the desired goal is achieved, it can decide whether it should continue to take similar actions or whether to pursue other paths.
- Unsupervised learning. Unsupervised learning is used when there are no specific labels you want to assign to your dataset or goal that you want to achieve, but you do want to understand how different observations in your dataset are related to each other. Clustering methods that aggregate records into groups of similar records are the most common type of supervised learning algorithms, but there are also other types of algorithms that fall into this category. For example, dimensionality reduction algorithms that reduce the number.of features in your dataset also fall into this category.
- When to use active learning
- When to use self supervised learning
- When to use semi supervised learning
- When to use weakly supervised learning
- When to use transfer learning