When to use function calling for LLMs

Are you wondering what function calling is for large language models (LLMs)? Or maybe you want to learn more about what types of situations you should use function calling in? Well either way, you are in the right place! In this article, we tell you everything you need to know to understand when to use function calling for LLMs.

We start out by talking about what function calling is in the context of LLMs and whether you need any specific data in order to use function calling. After that, we talk about some of the main advantages and disadvantages of function calling. Finally, we provide examples of situations where it does and does not make sense to use function calling.

What is function calling for large language models?

Function calling is a feature that was largely popularized by OpenAI that enables LLMs to extract structured data from unstructured text. The idea behind function calling is not that the LLM will necessarily call a function itself. Instead, the idea is that the LLM can extract the arguments that need to be passed to a function into a structured JSON object. This structured JSON object can then be passed to another function.

One thing that we want to emphasize up front is that function calling is not an inherent feature that all LLMs have. Instead, it is a feature that was originally popularized by a specific vendor that hosts LLM models that only applies to a subset of their models. That means that not all LLM models that you work with will be able to leverage function calling.

What data is needed for function calling?

Is there any data that you need to collect before you can use function calling for LLMs? The only information that you need to have available in order to implement function calling for LLMs is information on which attributes you want to extract from the unstructured task. You do not need to make any other data available in order to utilize function calling.

Advantages and disadvantages of function calling

What are some of the main advantages and disadvantages of function calling? In this section, we will discuss some of the main advantages and disadvantages of function calling. In particular, we will focus on advantages and disadvantages that function calling has compared to other methods that can be used to optimize systems that use LLMs.

Advantages of function calling

What are some of the main advantages of function calling? Here are some of the main advantages of function calling.

• Returns a consistent output format. One of the main advantages of function calling is that it consistently returns outputs in the same format. While it is possible to include information about how an output should be formatted in a standard prompt, there are not many consistency guarantees and there may be some situations where a model goes off the rails and returns an output that is not formatted correctly. Function calling can make it easier to create reliable systems with consistent performance.
• Relatively easy to implement. Another advantage of function calling is that it is relatively easy to implement. While there can be a small learning curve associated with understanding how to format information to function calls, function calling is very easy to implement once you get the hang of it. That means that even if function calling does not work well for your purposes, you did not waste much time.
• Relatively low latency. Another advantage of function calling is that it does not introduce too much additional latency into your system. While there may be some cases where the model takes longer to return a response, function calling does not require you to do things like make expensive calls to vector databases.
• Does not introduce privacy concerns. Since you do not need to feed much internal data to OpenAI in order to use function calling functionality, there are not large risks related to privacy. For example, you are not likely to find yourself in a situation where you have legal issues because your system accidentally exposed sensitive information to a user just because you are using function calling.
• Does not require large computational resources. Another advantage of function calling is that since you are not training a model or running large amounts of data through a model, you do not need to have access to large computational resources to implement function calling.

Disadvantages of function calling for LLMs

What are some of the main disadvantages of function calling for LLMs? Here are some of the main disadvantages of using function calling capabilities when working with LLMs.

• Can only be applied to some models. One of the main disadvantages of function calling is that function calling capabilities are only available for a small portion of models. If you need to be able to use function calling, you will not have as large of a selection of models and vendors to choose from.
• Can be non-intuitive. There can be a little bit of a learning curve associated with properly designing calls that use a function calling feature. This is especially true if you have never used a function calling feature before. Once you have used a function calling feature a few times, it becomes easier to design future calls.
• May not improve accuracy of response. While function calling does make it easy to ensure that your output is formatted correctly, it does not necessarily help to improve the veracity of the information that is contained in the output. If you are looking for techniques that can be used to improve the predictive performance of the model and ensure that it responds with logical information that is accurate, you may be better off looking into other methods.
• Cannot provide domain specific context. Another disadvantage of function calling is that it does not enable you to provide domain specific context that the model needs to operate effectively. If you need to provide your model with information that it would not otherwise have access to so that it can perform a specific task, you should look into using other methods.

When to use function calling for LLMs

What are some examples of situations where it makes sense to use function calling for LLMs? Here are some examples of situations where it makes sense to use function calling for LLMs.

• When you need to send the output of a LLM call to another function or API. In general, the function calling feature was designed with situations where you need to extract information from a document and pass that information to another function or API call in mind. That means that these types of situations are great use cases for function calling features.
• When you need to extract structured data from a freeform text document. More broadly, function calling can be used any time that you want to extract structured information from a freeform text document. For example if you want to extract a specific name or date from an unstructured piece of text, then this is a great use case for function calling. You should use function calling when you are operating in a situation where it is important that your outputs have a consistent format. While it is sometimes possible to get a LLM to return an output in a structured format without using function calling, there are not as many guarantees about the consistency of the output format.

When not to use function calling for LLMs

What are some examples of situations where you should not use function calling for LLMs? Here are some examples of situations where you may be better off looking into other techniques that can be used to optimize LLMs.

• When your goal is only to improve predictive performance. A lot of the magic of function calling comes from its ability to convert unstructured text to structured data. It is primarily used to ensure that models produce an appropriately formatted output, rather than to actually enhance the accuracy of the information in the output itself. If your main goal is to improve the predictive performance of your model rather than to ensure that the output is formatted correctly, you may be better off looking into techniques like retrieval augmented generation, prompt chaining, and basic prompt engineering.
• When you want to avoid vendor lock-in. Function calling capabilities are only built into a few models that are provided by one or two specific vendors. If you do not want to land yourself in a situation where you are tightly locked into using one specific vendor, then you may not want to leverage these capabilities.

Related articles

• How to improve LLM performance
• When to fine tune an LLM
• When to use retrieval augmented generation for LLMs
• When to use prompt chaining for LLMs
• When to use few shot learning for LLMs
• When to use basic prompt engineering for LLMs