Skip to main content
Structured outputs and tool calls are route-specific contracts. TokenLab forwards several public API formats, so the safest production pattern is to keep each provider-native tool shape on the route that owns it and validate model output in your own application.

​
Choose The Route

NeedRouteUse this shape
Portable JSON object responses/v1/chat/completionsresponse_format: {"type": "json_object"}
OpenAI-compatible function calling/v1/chat/completionstools: [{ "type": "function", "function": ... }]
OpenAI Responses tools/v1/responsesResponses tools, tool_choice, and text fields
Claude-native tool use or thinking/v1/messagesAnthropic Messages tool schema
Gemini function declarations or built-in tools/v1beta/models/:model:generateContentGemini-native tools and content parts
Do not send provider-native built-in tools through a different route and expect TokenLab to silently translate them.

​
JSON Mode

For portable structured responses, start with Chat Completions JSON mode: 
{
  "model": "gpt-5.4",
  "messages": [
    {
      "role": "user",
      "content": "Return a JSON object with city and weather."
    }
  ],
  "response_format": { "type": "json_object" }
}
The stable shared validation for Chat Completions accepts text and json_object. json_schema, strict, and provider-specific schema enforcement may exist for some upstream routes or conversion paths, but they are not a blanket promise across every TokenLab route and model. Verify them against the selected model before relying on them. Always parse and validate the returned JSON on your server. JSON mode improves the shape, but it does not replace application-level schema validation.

​
Tool Calling Loop

TokenLab does not execute your functions. Your application owns the loop:
  1. Send messages plus tool definitions.
  2. Read the model response for tool_calls, function_call, Anthropic tool_use, or Gemini function call parts.
  3. Execute the tool in your own backend.
  4. Append the tool result in the format required by the same route.
  5. Continue the conversation until the model returns a final answer.
Keep the same route throughout a tool loop. Mixing Chat Completions, Responses, Messages, and Gemini formats in one conversation usually creates subtle state and schema bugs.

​
Chat Completions Example

curl https://api.tokenlab.sh/v1/chat/completions \
  -H "Authorization: Bearer sk-your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "messages": [
      {
        "role": "user",
        "content": "Extract name and email as JSON. If needed, look up the customer."
      }
    ],
    "response_format": { "type": "json_object" },
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "lookup_customer",
          "description": "Look up a customer by email",
          "parameters": {
            "type": "object",
            "properties": {
              "email": { "type": "string" }
            },
            "required": ["email"]
          }
        }
      }
    ]
  }'

​
Schema Design

  • Keep schemas small and explicit. Large nested schemas add tokens and reduce reliability.
  • Prefer required fields for values your product cannot continue without.
  • Use enums for closed sets that your UI or backend depends on.
  • Include examples in the prompt when the model struggles with a shape.
  • Treat unsupported-field errors as contract feedback. Remove the field or use the native route that documents it.

​
Production Checklist

  • Record route, model, tool names, and sanitized schema shape in logs.
  • Validate tool arguments before executing any side effect.
  • Apply your own permission checks before tool execution.
  • Make tool execution idempotent when a client retry can repeat the same tool call.
  • Do not log secrets returned by tools into model-visible messages.

​
API Reference

TopicReference
Multi-Format APIMulti-Format API
Create Chat CompletionCreate Chat Completion
Create ResponseCreate Response
Create MessageCreate Message
Generate Gemini ContentGenerate Gemini Content