Tool calling

With tool calling (function calling), the model answers with a structured request to run one of your functions; you run it and send the result back. TOKEN accepts the OpenAI format on /v1/chat/completions and /v1/responses, and the Anthropic format on /v1/messages.

Models: every chat model in the catalog has tool calling except meta-llama/llama-4-maverick. The models page shows it per model (“tools”).

OpenAI format

Python

import json
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.thetoken.dev/v1", api_key=os.environ["TOKEN_API_KEY"])

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
messages = [{"role": "user", "content": "What's the weather in Riyadh?"}]

response = client.chat.completions.create(model="zai-org/glm-5.3", messages=messages, tools=tools)
message = response.choices[0].message

if message.tool_calls:
    messages.append(message)
    for call in message.tool_calls:
        args = json.loads(call.function.arguments)
        result = {"city": args["city"], "temperature_c": 31}  # call your real function here
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    response = client.chat.completions.create(model="zai-org/glm-5.3", messages=messages, tools=tools)

print(response.choices[0].message.content)

TypeScript

import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'https://api.thetoken.dev/v1', apiKey: process.env.TOKEN_API_KEY })

const tools: OpenAI.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Current weather for a city.',
      parameters: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
    },
  },
]
const messages: OpenAI.ChatCompletionMessageParam[] = [{ role: 'user', content: "What's the weather in Riyadh?" }]

let response = await client.chat.completions.create({ model: 'zai-org/glm-5.3', messages, tools })
const message = response.choices[0].message

if (message.tool_calls?.length) {
  messages.push(message)
  for (const call of message.tool_calls) {
    if (call.type !== 'function') continue
    const args = JSON.parse(call.function.arguments)
    const result = { city: args.city, temperature_c: 31 } // call your real function here
    messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) })
  }
  response = await client.chat.completions.create({ model: 'zai-org/glm-5.3', messages, tools })
}
console.log(response.choices[0].message.content)

The first response has finish_reason: "tool_calls" and a message like:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_7f3a9c",
      "type": "function",
      "function": { "name": "get_weather", "arguments": "{\"city\": \"Riyadh\"}" }
    }
  ]
}

tool_choice works as in OpenAI’s API: "auto" (the default), "none", "required", or {"type": "function", "function": {"name": "get_weather"}}.

Anthropic format

On /v1/messages, tools use Anthropic’s shape (name, description, input_schema), and the model answers with tool_use blocks:

curl https://api.thetoken.dev/v1/messages \
  -H "x-api-key: $TOKEN_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k3",
    "max_tokens": 1024,
    "tools": [{
      "name": "get_weather",
      "description": "Current weather for a city.",
      "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }],
    "messages": [{"role": "user", "content": "What is the weather in Riyadh?"}]
  }'

Send the result back as a tool_result block in the next user message, as with Anthropic’s API.

Models without tool calling

A request with tools to a model without tool calling is refused before it reaches a provider, with a 400 and nothing charged:

{
  "error": {
    "message": "meta-llama/llama-4-maverick does not support tool calling (function calling). Send the request without tools, or choose a model that supports them (the models page lists which do) (request id: 20261025091530123456789...)",
    "type": "new_api_error",
    "param": "",
    "code": "invalid_request"
  }
}

(The link in the message, to the models page, is shortened by the relay’s masking of URLs in error messages.) On /v1/messages the same message comes in Anthropic’s error format. Send the request without tools, or choose another model.