Are you wondering when to use recurrent neural networks? Then you are in the right place! In this article, we tell you everything you need to know to understand when to use recurrent neural networks.
We start out by discussing what types of outcome variables and features recurrent neural networks are generally used for. After that, we go over some of the main advantages and disadvantages of recurrent neural networks. Finally, we discuss some of the main scenarios where you should and should not use recurrent neural networks.
Types of outcomes for recurrent neural networks
Like other types of neural networks, recurrent neural networks are relatively flexible and can handle a wide variety of different outcome variables from simple tabular outcomes to more complex unstructured outcomes. What sets recurrent neural networks aside is that they can handle sequential data as both the input to the model and the output produced by the model. For example, they can handle text data made up of a sequence of words or time series data made up of an ordered sequence of measurements.
Here are a few examples of different prediction paradigms that recurrent neural networks can handle.
- Many-to-many: Many-to-many prediction problems are problems where both the input data for the model and the output data that is produced by the model consist of a sequence of multiple data points. An example of this would be if a model translated a sequence of words from English to Spanish.
- One-to-many: A one-to-many prediction problem is when there is a single datapoint used as an input to the model and a sequence of multiple data points is output from the model.
- Many-to-one: On the other hand, a many-to-one prediction problem is a problem where the model takes in a sequence of data points and outputs a single data point. One example of this would be if a model took in a sequence of words and outputted a single value representing whether the sentiment of the text was positive, negative or neutral.
Advantages and disadvantages of RNNs
What are some of the main advantages and disadvantages of recurrent neural networks? Here are some of the main advantages and disadvantages you should keep in mind when deciding when to use recurrent neural networks.
Advantages of recurrent neural networks
- Can handle a sequence of inputs. One advantage of recurrent neural networks is that they can take in a sequence of inputs. They account for the ordered nature of these inputs and can detect temporal trends that exist in the data.
- Can produce a sequence of outputs. On the flip side, recurrent neural networks can also be used if you need to produce a sequence of outputs for each row of input data. This can be useful for problems such as text generation where you need to produce an ordered sequence of outputs.
- No need to featurize data. Another advantage of recurrent neural networks is that they often eliminate the need to preprocess or featurize your data. For example, if you are using recurrent neural networks to make predictions on text data, you can feed the raw text straight into the model rather than having to create a TF-IDF matrix or apply a similar preprocessing step. This is a specific advantage that recurrent neural networks have over simpler text prediction paradigms.
Disadvantages of recurrent neural networks
- Require lots of data. Like other neural network models, recurrent neural network models require a lot of data to train. This poses a difficult challenge for tasks that do not come with built-in labels, such as some text prediction tasks.
- Computationally intensive. Another disadvantage of recurrent neural networks is that they are computationally intensive. Like many other neural networks, they require a lot of time and resources to train. They also generally also take more time to make predictions on new data points than simpler models.
- Many hyperparameters to tune. Recurrent neural networks are also sensitive to the choice of hyperparameters and model architecture used. This means that you may need to dedicate a large amount of time to properly configuring your model. Again, this is a downside that is common to all types of neural network models.
- Difficult to explain. Another disadvantage of recurrent neural networks is that they are difficult to explain to non-technical stakeholders. While even standard neural network architectures can be difficult to train, the sequential nature of recurrent neural networks adds some additional complexity.
- Not peak performance for text data. While recurrent neural networks used to be state of the art for text prediction problems, recent breakthroughs in the text analysis space have resulted in other model architectures that perform better. For example, transformer architectures tend to perform better on some problems.
When to use recurrent neural networks
When should you use recurrent neural networks over other machine learning models? Here are some of the main situations where you should use recurrent neural networks.
- Text and speech data. Since they can handle sequential inputs and outputs, recurrent neural networks are often used for problems that involve text and speech data. This includes models that take this kind of data as inputs as well as models that produce this kind of data as outputs. One caveat here is that if you are using text data to make predictions, you should look into text embedding models to see whether they are applicable for your use case (see the next section for more details).
- Time series data. Recurrent neural networks are also sometimes used for times series data. Because of the sequential nature of recurrent neural networks, they are able to detect temporal patterns in time series data.
- Any other sequential data. Recurrent neural networks can also be used in any other problems that have sequential data.
The not to use recurrent neural networks
When should you avoid using recurrent neural networks? Here are some examples of cases where you should avoid using recurrent neural networks.
- Simple tabular data. If you are working with simple tabular data that is not sequential in nature, you are better off avoiding neural networks. Many of the unique benefits of recurrent neural networks are lost in this paradigm.
- Image data and spatial patterns. Recurrent neural networks are good at detecting temporal patterns in your data, but if you have data with spatial patterns then you are better off using a convolutional neural network. Image data is a common example of one type of data that has spatial patterns in it.
- Text embeddings would suffice. If you are planning on using a recurrent neural network for a many-to-one problem with text input data, you should assess whether using a text embedding model would suffice for your purposes. A text embedding model is a pre-trained model that can be used to convert sequences of text into a numeric representation that can then be fed to a simple model designed for tabular data. Since text embedding models come pre-trained, you can avoid some of the common issues associated with training a recurrent neural network from scratch such as the large data requirements and long training times.
Related articles
Are you trying to figure out which machine learning model is best for your next data science project? Check out our comprehensive guide on how to choose the right machine learning model.
