LiteLLM

Overview

LiteLLM fits TokenLab in two common ways:

use TokenLab as an OpenAI-compatible endpoint behind LiteLLM
put LiteLLM in front of TokenLab when your team wants team-managed virtual keys, model-selection policy, or centralized observability

For TokenLab, the cleanest default is LiteLLM’s custom OpenAI / OpenAI-compatible path pointed at https://api.tokenlab.sh/v1.

If you specifically need Claude-native or Gemini-native request shapes, prefer TokenLab’s dedicated native integrations instead of forcing those workflows through LiteLLM’s OpenAI-compatible abstraction.

Type: Framework or PlatformPrimary Path: OpenAI-compatible endpointSupport Confidence: Supported path

Install

pip install 'litellm[proxy]'

Proxy Configuration

Create a litellm-config.yaml like this:

model_list:
  - model_name: tokenlab-gpt-5.4
    litellm_params:
      model: custom_openai/gpt-5.4
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY

  - model_name: tokenlab-claude-sonnet
    litellm_params:
      model: custom_openai/claude-sonnet-4-6
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY

Start the proxy:

export OPENAI_API_KEY="sk-your-tokenlab-key"
litellm --config litellm-config.yaml --port 4000

Call LiteLLM Through OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="anything",
    base_url="http://127.0.0.1:4000"
)

response = client.chat.completions.create(
    model="tokenlab-gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

Direct Python Usage

If you are using LiteLLM as a Python library instead of the proxy, keep the same TokenLab base URL:

import litellm

response = litellm.completion(
    model="custom_openai/gpt-5.4",
    api_base="https://api.tokenlab.sh/v1",
    api_key="sk-your-tokenlab-key",
    messages=[{"role": "user", "content": "Summarize this repo."}]
)

Best Practices

Prefer custom_openai for TokenLab

Treat TokenLab as an OpenAI-compatible endpoint unless you have a very specific reason to build a more complex provider mapping.

Use LiteLLM when you need one more gateway layer

LiteLLM makes sense when your own platform wants virtual keys, extra model-selection policy, or centralized logs in front of TokenLab.

Keep native-provider expectations realistic

OpenAI-compatible translation layers are great for broad compatibility, but they are not the right place to promise every provider-native feature.

Troubleshooting

Connection errors

Verify api_base is exactly https://api.tokenlab.sh/v1
Make sure LiteLLM can reach TokenLab over the public internet
If you run the proxy locally, verify the OpenAI client points to your LiteLLM port instead of TokenLab directly

Authentication errors

Check that LiteLLM is reading the right OPENAI_API_KEY
Confirm the TokenLab key starts with sk-
Confirm the key is active in TokenLab dashboard

Model not found

Verify the TokenLab model name in custom_openai/<model>
Keep your LiteLLM model_name alias separate from the real TokenLab model id

Compatibility Matrix

Coding Tools

Native SDKs

Frameworks & Platforms

Chat Apps

Overview

Install

Proxy Configuration

Call LiteLLM Through OpenAI SDK

Direct Python Usage

Best Practices

Troubleshooting

​Overview

​Install

​Proxy Configuration

​Call LiteLLM Through OpenAI SDK

​Direct Python Usage

​Best Practices

​Troubleshooting

Overview

Install

Proxy Configuration

Call LiteLLM Through OpenAI SDK

Direct Python Usage

Best Practices

Troubleshooting