Skip to main content

​
Overview

LiteLLM fits TokenLab in two common ways:
  • use TokenLab as an OpenAI-compatible endpoint behind LiteLLM
  • put LiteLLM in front of TokenLab when your team wants team-managed virtual keys, model-selection policy, or centralized observability
For TokenLab, the cleanest default is LiteLLM’s custom OpenAI / OpenAI-compatible path pointed at https://api.tokenlab.sh/v1.
If you specifically need Claude-native or Gemini-native request shapes, prefer TokenLab’s dedicated native integrations instead of forcing those workflows through LiteLLM’s OpenAI-compatible abstraction.
Type: Framework or PlatformPrimary Path: OpenAI-compatible endpointSupport Confidence: Supported path

​
Install

pip install 'litellm[proxy]'

​
Proxy Configuration

Create a litellm-config.yaml like this: 
model_list:
  - model_name: tokenlab-gpt-5.4
    litellm_params:
      model: custom_openai/gpt-5.4
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY

  - model_name: tokenlab-claude-sonnet
    litellm_params:
      model: custom_openai/claude-sonnet-4-6
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY
Start the proxy: 
export OPENAI_API_KEY="sk-your-tokenlab-key"
litellm --config litellm-config.yaml --port 4000

​
Call LiteLLM Through OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="anything",
    base_url="http://127.0.0.1:4000"
)

response = client.chat.completions.create(
    model="tokenlab-gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

​
Direct Python Usage

If you are using LiteLLM as a Python library instead of the proxy, keep the same TokenLab base URL: 
import litellm

response = litellm.completion(
    model="custom_openai/gpt-5.4",
    api_base="https://api.tokenlab.sh/v1",
    api_key="sk-your-tokenlab-key",
    messages=[{"role": "user", "content": "Summarize this repo."}]
)

​
Best Practices

Treat TokenLab as an OpenAI-compatible endpoint unless you have a very specific reason to build a more complex provider mapping.
LiteLLM makes sense when your own platform wants virtual keys, extra model-selection policy, or centralized logs in front of TokenLab.
OpenAI-compatible translation layers are great for broad compatibility, but they are not the right place to promise every provider-native feature.

​
Troubleshooting

  • Verify api_base is exactly https://api.tokenlab.sh/v1
  • Make sure LiteLLM can reach TokenLab over the public internet
  • If you run the proxy locally, verify the OpenAI client points to your LiteLLM port instead of TokenLab directly
  • Check that LiteLLM is reading the right OPENAI_API_KEY
  • Confirm the TokenLab key starts with sk-
  • Confirm the key is active in TokenLab dashboard
  • Verify the TokenLab model name in custom_openai/<model>
  • Keep your LiteLLM model_name alias separate from the real TokenLab model id