Are you wondering when you should use multinomial regression over another machine learning model? Or maybe you want to hear more about when to use multinomial regression and when to use ordinal logistic regression. Well either way, you are in the right place! In this article we tell you everything you need to know to determine when to use multinomial regression.
This article starts out with a discussion of what outcome variables can be handled using multinomial regression. After that, we discuss some of the main advantages and disadvantages you should keep in mind when deciding whether to use multinomial regression. Finally, we discuss some specific examples of situations where you should and should not use multinomial regression.
What outcomes can multinomial regression handle?
What kind of outcome variables can multinomial regression handle? Multinomial regression is intended to be used when you have a categorical outcome variable that has more than 2 levels. Multinomial regression is generally intended to be used for outcome variables that have no natural ordering to them. If you have a multiclass outcome variable such that the classes have a natural ordering to them, you should look into whether ordinal logistic regression would be more well suited for your purpose.
Advantages and disadvantages of multinomial regression
So what are the main advantages and disadvantages of multinomial regression? Here are some of the main advantages and disadvantages you should keep in mind when deciding whether to use multinomial regression.
Advantages of multinomial regression
- Interpretable coefficients. One of the main advantages of multinomial regression is that it provides highly interpretable coefficients that quantify the relationship between your features and your outcome variable. There are not many other models that provide this level of interpretability for multiclass outcomes.
- More flexible than ordinal logistic regression. Another benefit that multinomial logistic regression hass over ordinal logistic regression is that multinomial logistic regression is a more flexible model than ordinal logistic regression. That means that it does not make as many strong assumptions about the structure of your data.
Disadvantages of multinomial regression
- Cannot address ordered variables. One disadvantage of multinomial regression is that it can not account for multiclass outcome variables that have a natural ordering to them. You can still use multinomial regression in these types of scenarios, but it will not account for any natural ordering between the levels of those variables.
- More parameters than multinomial logistic regression. Another disadvantage that multinomial regression has compared to ordinal logistic regression is that multinomial regression models have many more parameters that need to be estimated than ordinal logistic regression models. These additional parameters are what allows the multinomial regression model to have more flexibility than the ordinal logistic regression model.
- Not available in some libraries. Another disadvantage of multinomial logistic regression is that it is a relatively niche model that is not available in all common machine learning libraries. Since regression models like multinomial regression are more commonly used for inference than prediction, it is more common for multinomial logistic regression to be available in packages like SAS and Stata that have a heavier focus on classical statistical models and inference.
- General regression downsides. Multinomial regression is also affected by some of the common pitfalls that affect other regression models like linear regression and logistic regression. For example, multinomial regression models are affected by issues like outliers, correlated features, missing data, and unspecified interactions.
When to use multinomial regression
So when should you use multinomial logistic regression? Here are some examples of scenarios where you should use multinomial logistic regression.
- Multiclass outcome and inference is your primary goal. If you have a multiclass outcome variable and you are more interested in inference than prediction, you should almost certainly reach for a multiclass regression model such as multinomial regression or ordinal logistic regression. Like other regression models, these models provide interpretable coefficients that quantify the relationship between your features and your outcome variable.
- Ordinal outcome that does not meet proportional odds assumption. Even if you are using an outcome variable that has a natural order to it, you may be better off using multinomial regression in some cases. This is because ordinal logistic regression models make a fairly strong assumption called the proportional odds assumption. This assumption essentially implies that the differences associated with moving from one category of the outcome variable to the next higher category are the same across all categories. This is a fairly stringent assumption that does not apply in all cases and if you do not think this assumption applies for your use case, you may be better off treating your outcome variable as an unordered variable and using multinomial logistic regression instead.
When not to use multinomial regression
When should you avoid using multinomial logistic regression? Here are some examples of scenarios where you should avoid using multinomial logistic regression.
- Naturally ordered outcome variable. In general, multinomial regression is intended to be used when you have a multiclass outcome variable that does not have a natural order to it. If your outcome variable has a natural order to it, you should look into whether ordinal logistic regression is more suitable for your purposes. For example, if your outcome variable is a survey response that rates the level of trust a person had in a certain politician as very low, low, medium, high, or very high, this would be an example of a multiclass outcome with a natural order to it. In this case, low is higher than very low, medium is higher than low, and so on.
- Ordered outcomes with a small sample size. Multinomial logistic regression models have many more parameters that need to be estimated than ordinal logistic regression models. In general, this means that you will need more data to get good estimates of these parameters. That means that ordinal regression models generally perform better on smaller datasets when they are applicable.
Related articles
- When to use ordinal logistic regression
- When to use linear regression
- When to use logistic regression
- When to use poisson regression
- When to use Bayesian regression
- When to use ridge regression
- When to use LASSO
- When to use mixed models
Are you trying to figure out which machine learning model is best for your next data science project? Check out our comprehensive guide on how to choose the right machine learning model.
