Skip to main content

Tool & Function Calling

Tool calls (also known as function calls) enable LLMs to access external tools. The LLM does not directly call these tools but instead suggests which tool to invoke. The user then independently calls the tool and returns the result to the LLM. Finally, the LLM formats the response as an answer to the user's original question.

Knox Chat standardizes the tool-calling interface across models and providers.

For a primer on how tool calls work in the OpenAI SDK, refer to this article. If you prefer learning from a full end-to-end example, continue reading below.

Tool Calling Example​

Below are Python and TypeScript example codes that enable the LLM to call an external API—in this case, Project Gutenberg—to search for books.

import json, requests
from openai import OpenAI

KNOXCHAT_API_KEY = f"<KNOXCHAT_API_KEY>"

# You can use any model that supports tool calling
MODEL = "google/gemini-2.0-flash-001"

openai_client = OpenAI(
  base_url="https://knox.chat/v1",
  api_key=KNOXCHAT_API_KEY,
)

task = "What are the titles of some James Joyce books?"

messages = [
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": task,
  }
]

Defining the Tool​

Next, we define the tool to be called. Keep in mind that this tool will receive the request from the LLM, but the code we write here will ultimately be responsible for executing the call and returning the result to the LLM.

def search_gutenberg_books(search_terms):
    search_query = " ".join(search_terms)
    url = "https://gutendex.com/books"
    response = requests.get(url, params={"search": search_query})

    simplified_results = []
    for book in response.json().get("results", []):
        simplified_results.append({
            "id": book.get("id"),
            "title": book.get("title"),
            "authors": book.get("authors")
        })

    return simplified_results

tools = [
  {
    "type": "function",
    "function": {
      "name": "search_gutenberg_books",
      "description": "Search for books in the Project Gutenberg library based on specified search terms",
      "parameters": {
        "type": "object",
        "properties": {
          "search_terms": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "description": "List of search terms to find books in the Gutenberg library (e.g. ['dickens', 'great'] to search for books by Dickens with 'great' in the title)"
          }
        },
        "required": ["search_terms"]
      }
    }
  }
]

TOOL_MAPPING = {
    "search_gutenberg_books": search_gutenberg_books
}

Note that a "tool" is just a regular function. Then, we write a JSON "specification" compatible with OpenAI's function calling parameters. We pass this specification to the LLM so it knows this tool is available and how to use it. The LLM will request the tool when needed, along with any parameters. Then, we will locally marshal the tool call, execute the function, and return the result to the LLM.

Tool Usage and Tool Call Results​

Let's make the first Knox Chat API call to this model:

request_1 = {
    "model": google/gemini-2.0-flash-001,
    "tools": tools,
    "messages": messages
}

response_1 = openai_client.chat.completions.create(**request_1).message

The LLM responds with a completion reason and an array of tool_calls. In a general LLM response handler, you would want to check the completion reason before processing the tool calls, but here we'll assume that's indeed the case. Let's proceed with handling the tool calls:

# Append the response to the messages array so the LLM has the full context
# It's easy to forget this step!
messages.append(response_1)

# Now we process the requested tool calls, and use our book lookup tool
for tool_call in response_1.tool_calls:
    '''
    In this case we only provided one tool, so we know what function to call.
    When providing multiple tools, you can inspect `tool_call.function.name`
    to figure out what function you need to call locally.
    '''
    tool_name = tool_call.function.name
    tool_args = json.loads(tool_call.function.arguments)
    tool_response = TOOL_MAPPING[tool_name](**tool_args)
    messages.append({
      "role": "tool",
      "tool_call_id": tool_call.id,
      "name": tool_name,
      "content": json.dumps(tool_response),
    })

The message array now contains:

  1. Our original request
  2. The LLM's response (including the tool call request)
  3. The tool call result (the JSON object returned from the Project Gutenberg API)

Now, we can proceed with the second Knox Chat API call and expect the result!

request_2 = {
  "model": MODEL,
  "messages": messages,
  "tools": tools
}

response_2 = openai_client.chat.completions.create(**request_2)

print(response_2.choices[0].message.content)

The output will be displayed similarly as follows:

Here are some books by James Joyce:

*   *Ulysses*
*   *Dubliners*
*   *A Portrait of the Artist as a Young Man*
*   *Chamber Music*
*   *Exiles: A Play in Three Acts*

We did it! We successfully used tools in the prompt.

A Simple Agent Loop​

In the example above, the calls were explicit and sequential. To handle a variety of user inputs and tool calls, you can use an agent loop.

Here's an example of a simple agent loop (using the same "tools" and initial "message" as above):


def call_llm(msgs):
    resp = openai_client.chat.completions.create(
        model=google/gemini-2.0-flash-001,
        tools=tools,
        messages=msgs
    )
    msgs.append(resp.choices[0].message.dict())
    return resp

def get_tool_response(response):
    tool_call = response.choices[0].message.tool_calls[0]
    tool_name = tool_call.function.name
    tool_args = json.loads(tool_call.function.arguments)

    # Look up the correct tool locally, and call it with the provided arguments
    # Other tools can be added without changing the agentic loop
    tool_result = TOOL_MAPPING[tool_name](**tool_args)

    return {
        "role": "tool",
        "tool_call_id": tool_call.id,
        "name": tool_name,
        "content": tool_result,
    }

while True:
    resp = call_llm(_messages)

    if resp.choices[0].message.tool_calls is not None:
        messages.append(get_tool_response(resp))
    else:
        break

print(messages[-1]['content'])