Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.tokenlab.sh/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

LiteLLM fits TokenLab in two common ways:
  • use TokenLab as an OpenAI-compatible upstream behind LiteLLM
  • put LiteLLM in front of TokenLab when your team wants one more internal gateway layer for routing, virtual keys, or centralized observability
For TokenLab, the cleanest default is LiteLLM’s custom OpenAI / OpenAI-compatible path pointed at https://api.tokenlab.sh/v1.
If you specifically need Claude-native or Gemini-native request shapes, prefer TokenLab’s dedicated native integrations instead of forcing those workflows through LiteLLM’s OpenAI-compatible abstraction.
Type: Framework or PlatformPrimary Path: OpenAI-compatible upstreamSupport Confidence: Supported path

​
Install

pip install 'litellm[proxy]'

​
Proxy Configuration

Create a litellm-config.yaml like this: 
model_list:
  - model_name: tokenlab-gpt-5.4
    litellm_params:
      model: custom_openai/gpt-5.4
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY

  - model_name: tokenlab-claude-sonnet
    litellm_params:
      model: custom_openai/claude-sonnet-4-6
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY
Start the proxy: 
export OPENAI_API_KEY="sk-your-tokenlab-key"
litellm --config litellm-config.yaml --port 4000

​
Call LiteLLM Through OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="anything",
    base_url="http://127.0.0.1:4000"
)

response = client.chat.completions.create(
    model="tokenlab-gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

​
Direct Python Usage

If you are using LiteLLM as a Python library instead of the proxy, keep the same TokenLab base URL: 
import litellm

response = litellm.completion(
    model="custom_openai/gpt-5.4",
    api_base="https://api.tokenlab.sh/v1",
    api_key="sk-your-tokenlab-key",
    messages=[{"role": "user", "content": "Summarize this repo."}]
)

​
Best Practices

Treat TokenLab as an OpenAI-compatible upstream unless you have a very specific reason to build a more complex provider mapping.
LiteLLM makes sense when your own platform wants virtual keys, extra routing policy, or centralized logs in front of TokenLab.
OpenAI-compatible translation layers are great for broad compatibility, but they are not the right place to promise every provider-native feature.

​
Troubleshooting

  • Verify api_base is exactly https://api.tokenlab.sh/v1
  • Make sure LiteLLM can reach TokenLab over the public internet
  • If you run the proxy locally, verify the OpenAI client points to your LiteLLM port instead of TokenLab directly
  • Check that LiteLLM is reading the right OPENAI_API_KEY
  • Confirm the TokenLab key starts with sk-
  • Confirm the key is active in TokenLab dashboard
  • Verify the TokenLab model name in custom_openai/<model>
  • Keep your LiteLLM model_name alias separate from the real TokenLab model id