IDE & SDK Compatibility

Overview

TokenLab exposes multiple API formats so common coding tools, SDKs, and frameworks can integrate with minimal glue code. This page is intentionally narrower than a marketing matrix:

Supported means we document a concrete setup path and TokenLab exposes the protocol shape that path expects.
Strong native path means the repo also has direct adapter or request-format evidence for that protocol family.
Best-effort means the integration can work, but the upstream client does not treat this custom gateway workflow as a stable contract.

Unsupported fields are not handled uniformly. On compatibility routes, some fields are ignored or normalized. On /v1/responses, unsupported fields can return explicit 400 or 503 errors when that route cannot guarantee the requested behavior.

Supported API Formats

Endpoint	Format	Use Case
`/v1/chat/completions`	OpenAI Chat	Universal compatibility
`/v1/responses`	OpenAI Responses	Stateful conversations
`/v1/messages`	Anthropic Messages	Claude native features
`/v1beta/models/:model:generateContent`	Google Gemini	Gemini native features

IDE & CLI Compatibility

Documented Tool Paths

Tool	Support Level	Format	Notes
Cursor	Supported with limits	OpenAI-compatible	Works for BYOK standard chat/editor flows, not as a replacement for Cursor-managed features like Tab Completion
Claude Code CLI	Strong native path	Anthropic	Native `/v1/messages` route with adapter coverage for thinking and `tool_choice`
Codex CLI	Supported with model/path limits	OpenAI Responses	Treat `/v1/responses` as an advanced path for Codex-specific workflows; some Responses-only fields are not guaranteed across every model and routed path
Gemini CLI	Best-effort / experimental	Gemini	Custom TokenLab base URL flow is not an official Gemini CLI integration
OpenCode	Supported	OpenAI-compatible	Use an OpenAI-compatible provider by default; move to a Responses-based provider only when you explicitly need it
OpenHands	Supported	OpenAI-compatible	Use the LiteLLM/OpenAI-compatible path with `openai/<model>` and TokenLab’s `/v1` base URL

Other OpenAI-compatible editors and agent tools often work with the same base URL pattern; check the tool’s own custom-provider support before using it in production.

Configuration Examples

Cursor
Claude Code
OpenCode
Aider

Base URL: https://api.tokenlab.sh/v1
API Key: sk-your-tokenlab-key

Cursor uses Anthropic-style tool format internally. TokenLab supports both:

OpenAI format: { type: "function", function: { name, parameters } }
Anthropic format: { name, input_schema } (no type field)

export ANTHROPIC_BASE_URL="https://api.tokenlab.sh"
export ANTHROPIC_API_KEY="sk-your-tokenlab-key"

export OPENAI_API_KEY="sk-your-tokenlab-key"
export LOCAL_ENDPOINT="https://api.tokenlab.sh/v1"

export OPENAI_API_KEY="sk-your-tokenlab-key"
export OPENAI_BASE_URL="https://api.tokenlab.sh/v1"
aider --model gpt-5.4

SDK Compatibility

Documented SDK & Framework Paths

SDK / Framework	Language	Support Level	Notes
OpenAI SDK	Python/JS/Go	Supported core path	Compatibility path for `/v1` Chat Completions and Embeddings; use native provider routes for provider-specific behavior, and do not assume Responses-only fields work across every model
Anthropic SDK	Python/JS	Strong native path	Native Messages route with direct evidence for tools, thinking, and prompt caching
Vercel AI SDK	TypeScript	Recommended integration pattern	Prefer `@ai-sdk/openai-compatible`; use `@ai-sdk/openai` only when you explicitly want Responses-native behavior
LangChain	Python/JS	Supported standard surfaces	`ChatOpenAI` and `OpenAIEmbeddings` are the intended scope; vendor-native extras are out of scope
LlamaIndex	Python	Supported via `OpenAILike`	Use `OpenAILike`, not the built-in OpenAI classes, for third-party gateways such as TokenLab
OpenAI Agents SDK	Python	Supported chat-completions path	Use `AsyncOpenAI(base_url="https://api.tokenlab.sh/v1")` with `OpenAIChatCompletionsModel`; native Responses behavior depends on the selected model and route
AutoGen	Python	Supported custom endpoint path	Use `OpenAIChatCompletionClient` with `base_url: https://api.tokenlab.sh/v1`
Semantic Kernel	.NET	Supported custom endpoint path	Use the OpenAI chat completion connector custom endpoint and point it at `https://api.tokenlab.sh/v1`
CrewAI	Python	Supported custom endpoint path	Set `OPENAI_API_BASE` or `LLM(base_url=...)` to TokenLab’s `/v1` endpoint
Pydantic AI	Python	Supported OpenAI-compatible provider path	Use `OpenAIChatModel` with `OpenAIProvider(base_url="https://api.tokenlab.sh/v1")`
DSPy	Python	Supported custom endpoint path	Use `dspy.LM("openai/<model>", api_base="https://api.tokenlab.sh/v1")`
Langflow	Python / Web	Supported with scope limits	Use the OpenAI component’s OpenAI API Base field with TokenLab’s `/v1` endpoint
Haystack	Python	Supported custom endpoint path	Use `OpenAIChatGenerator(api_base_url="https://api.tokenlab.sh/v1")`
Graphiti	Python	Supported custom endpoint path	Use `OpenAIGenericClient` with TokenLab’s `/v1` endpoint
Dify	-	Supported with scope limits	OpenAI provider and chat-completions-oriented flows are the intended path; not a fit for Codex-specific Responses or WebSocket behavior
Flowise	Web / Node	Supported with scope limits	Use the TokenLab chat node when available, or a ChatOpenAI-compatible node pointed at `https://api.tokenlab.sh/v1`
Mem0	Python	Supported custom endpoint path	Keep the OpenAI provider and set `openai_base_url` to TokenLab’s `/v1` endpoint
Agno	Python	Supported path	Use the TokenLab model wrapper when available, or the OpenAI-compatible base URL path
Browser Use	Python	Supported custom endpoint path	Use `ChatOpenAILike` with TokenLab’s `/v1` endpoint
VoltAgent	TypeScript	Supported custom endpoint path	Pass an AI SDK OpenAI provider configured with `baseURL: "https://api.tokenlab.sh/v1"`
Ragas	Python	Supported custom endpoint path	Pass `AsyncOpenAI(base_url="https://api.tokenlab.sh/v1")` into `llm_factory`
Guardrails	Python	Supported validation path	Pass an OpenAI SDK chat completions callable configured with TokenLab’s `/v1` endpoint
Prompt flow	Python / CLI	Supported custom endpoint path	Create an OpenAI connection with `base_url=https://api.tokenlab.sh/v1`
Promptfoo	CLI / Node	Supported path	Use OpenAI chat or Responses providers with `apiBaseUrl: https://api.tokenlab.sh/v1`
Portkey Gateway	Gateway	Supported path	Use TokenLab as an upstream provider for OpenAI-compatible chat and Responses traffic
Helicone	Gateway / Observability	Supported observability path	Route OpenAI-compatible requests through Helicone Gateway with `Helicone-Target-Url: https://api.tokenlab.sh`
Langfuse	Observability	Supported tracing path	Configure the Langfuse OpenAI integration with TokenLab’s `/v1` base URL
OpenLIT	Observability	Supported tracing path	Instrument an OpenAI SDK client configured with TokenLab’s `/v1` base URL
OpenLLMetry	Observability	Supported tracing path	Use OpenAI SDK instrumentation with TokenLab’s `/v1` base URL
Phoenix	Observability	Supported tracing path	Auto-instrument the OpenAI SDK client configured with TokenLab’s `/v1` base URL
Opik	Observability	Supported tracing path	Wrap an OpenAI SDK client configured with TokenLab’s `/v1` base URL
LangBot	Bot Platform	Supported chat path	Use the TokenLab requester when available, or configure an OpenAI-compatible requester with TokenLab’s `/v1` endpoint
Open WebUI	Web Chat	Supported chat-app path	Add TokenLab as an OpenAI-compatible API connection and refresh the model list
Chatbox / DeepChat / Jan / LibreChat / Cherry Studio	Desktop / Web	Supported chat-app path	Use each app’s OpenAI-compatible custom provider flow; these apps are usually chat-completions oriented

Chat Completions Parameters

Core Parameters

Parameter	Type	Description
`model`	string	Model identifier (required)
`messages`	array	Conversation messages (required)
`max_tokens`	integer	Maximum output tokens
`temperature`	number	Sampling temperature (0-2)
`top_p`	number	Nucleus sampling (0-1)
`stream`	boolean	Enable streaming

Tool Calling

{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": { "type": "string" }
          }
        },
        "strict": true
      }
    }
  ],
  "tool_choice": "auto",
  "parallel_tool_calls": true
}

Tool Choice Options

Format	Example	Description
String	`"auto"`, `"none"`, `"required"`	Simple selection
OpenAI Object	`{ "type": "function", "function": { "name": "fn" } }`	Force specific function
Anthropic Object	`{ "type": "tool", "name": "fn", "disable_parallel_tool_use": true }`	Anthropic native format

Advanced Parameters

Parameter	Type	Description
`stream_options`	object	Chat Completions only: `{ include_usage: true }` for token counts
`reasoning_effort`	string	`"low"`, `"medium"`, `"high"` for reasoning-enabled GPT-5 models
`service_tier`	string	`"auto"` or `"default"`
`seed`	integer	Deterministic outputs
`logprobs`	boolean	Return log probabilities
`top_logprobs`	integer	Number of top logprobs (0-20)
`logit_bias`	object	Token bias map (-100 to 100)
`frequency_penalty`	number	Repetition penalty (-2 to 2)
`presence_penalty`	number	Topic penalty (-2 to 2)
`stop`	string/array	Stop sequences
`n`	integer	Number of completions (1-128)
`user`	string	User identifier for tracking

OpenAI Advanced Features

Parameter	Type	Description
`modalities`	array	`["text", "audio"]` for multimodal
`audio`	object	Audio output config (voice, format)
`prediction`	object	Predicted output for faster completion
`metadata`	object	Key-value pairs for tracking
`store`	boolean	Store for later retrieval

Provider-Specific Options

{
  "anthropic_options": {
    "thinking": {
      "type": "enabled",
      "budget_tokens": 10000
    },
    "prompt_caching": true
  },
  "google_options": {
    "safety_settings": [...],
    "google_search": true,
    "code_execution": true
  }
}

Anthropic Messages Parameters

Core Parameters

Parameter	Type	Description
`model`	string	Model identifier
`messages`	array	Conversation messages
`max_tokens`	integer	Maximum output (up to 128000)
`system`	string/array	System prompt
`stream`	boolean	Enable streaming

Tool Calling

{
  "tools": [
    {
      "name": "get_weather",
      "description": "Get weather",
      "input_schema": {
        "type": "object",
        "properties": {
          "location": { "type": "string" }
        }
      }
    }
  ],
  "tool_choice": {
    "type": "auto",
    "disable_parallel_tool_use": false
  }
}

Extended Thinking

{
  "model": "claude-opus-4-6",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 10000
  }
}

Responses API Parameters

Core Parameters

Parameter	Type	Description
`model`	string	Model identifier
`input`	string/array	Input content
`instructions`	string	System instructions
`max_output_tokens`	integer	Maximum output tokens
`previous_response_id`	string	Continue conversation

Advanced Parameters

Parameter	Type	Description
`truncation_strategy`	string	`"auto"` or `"disabled"`
`include`	array	`["reasoning.encrypted_content"]`
`reasoning_effort`	string	For reasoning models
`service_tier`	string	Priority tier

Tool Format

Supports both OpenAI and Anthropic tool formats:

// OpenAI format
{ "type": "function", "name": "fn", "parameters": {...} }

// Anthropic format (Cursor compatibility)
{ "name": "fn", "input_schema": {...} }

Gemini API Parameters

Core Parameters

Parameter	Type	Description
`contents`	array	Conversation content
`systemInstruction`	object	System prompt
`generationConfig`	object	Generation settings

Tools

{
  "tools": [{
    "functionDeclarations": [{
      "name": "search",
      "description": "Search the web",
      "parameters": {...}
    }],
    "codeExecution": {},
    "googleSearch": {}
  }],
  "toolConfig": {
    "functionCallingConfig": {
      "mode": "AUTO"
    }
  }
}

Safety Settings

{
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_MEDIUM_AND_ABOVE"
    }
  ]
}

Additional Parameters

Parameter	Type	Description
`cachedContent`	string	Cached content reference
`responseMimeType`	string	`"text/plain"` or `"application/json"`
`responseSchema`	object	JSON schema for structured output

Streaming

Generation endpoints that expose stream: true, including Chat Completions and Responses, use Server-Sent Events (SSE):

# Chat Completions
curl https://api.tokenlab.sh/v1/chat/completions \
  -H "Authorization: Bearer sk-xxx" \
  -d '{"model": "gpt-4o", "messages": [...], "stream": true}'

# Chat Completions usage tracking
-d '{"...", "stream_options": {"include_usage": true}}'

Error Handling

TokenLab returns OpenAI-compatible error responses:

{
  "error": {
    "message": "Invalid API key",
    "type": "invalid_api_key",
    "code": "invalid_api_key"
  }
}

See Error Handling Guide for details.

Best Practices

Use passthrough for unknown parameters

Unknown parameters are forwarded only when the selected public route and model support them.

Use stream_options.include_usage only for Chat Completions

For Chat Completions streaming, enable stream_options.include_usage for accurate token counts. Responses has its own streaming contract and does not expose this Chat-only option.

Use appropriate tool_choice format

Match your SDK’s expected format. TokenLab accepts both OpenAI and Anthropic formats.

​Overview

​Supported API Formats

​IDE & CLI Compatibility

​Documented Tool Paths

​Configuration Examples

​SDK Compatibility

​Documented SDK & Framework Paths

​Chat Completions Parameters

​Core Parameters

​Tool Calling

​Tool Choice Options

​Advanced Parameters

​OpenAI Advanced Features

​Provider-Specific Options

​Anthropic Messages Parameters

​Core Parameters

​Tool Calling

​Extended Thinking

​Responses API Parameters

​Core Parameters

​Advanced Parameters

​Tool Format

​Gemini API Parameters

​Core Parameters

​Tools

​Safety Settings

​Additional Parameters

​Streaming

​Error Handling

​Best Practices

Overview

Supported API Formats

IDE & CLI Compatibility

Documented Tool Paths

Configuration Examples

SDK Compatibility

Documented SDK & Framework Paths

Chat Completions Parameters

Core Parameters

Tool Calling

Tool Choice Options

Advanced Parameters

OpenAI Advanced Features

Provider-Specific Options

Anthropic Messages Parameters

Core Parameters

Tool Calling

Extended Thinking

Responses API Parameters

Core Parameters

Advanced Parameters

Tool Format

Gemini API Parameters

Core Parameters

Tools

Safety Settings

Additional Parameters

Streaming

Error Handling

Best Practices