> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokenlab.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Use TokenLab with LiteLLM as an OpenAI-compatible endpoint or as part of your team gateway

## Overview

LiteLLM fits TokenLab in two common ways:

* use TokenLab as an **OpenAI-compatible endpoint** behind LiteLLM
* put LiteLLM in front of TokenLab when your team wants team-managed virtual keys, model-selection policy, or centralized observability

For TokenLab, the cleanest default is LiteLLM's **custom OpenAI / OpenAI-compatible** path pointed at `https://api.tokenlab.sh/v1`.

<Note>
  If you specifically need Claude-native or Gemini-native request shapes, prefer TokenLab's dedicated native integrations instead of forcing those workflows through LiteLLM's OpenAI-compatible abstraction.
</Note>

<Note>
  **Type**: Framework or Platform

  **Primary Path**: OpenAI-compatible endpoint

  **Support Confidence**: Supported path
</Note>

## Install

```bash theme={null}
pip install 'litellm[proxy]'
```

## Proxy Configuration

Create a `litellm-config.yaml` like this:

```yaml theme={null}
model_list:
  - model_name: tokenlab-gpt-5.4
    litellm_params:
      model: custom_openai/gpt-5.4
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY

  - model_name: tokenlab-claude-sonnet
    litellm_params:
      model: custom_openai/claude-sonnet-4-6
      api_base: https://api.tokenlab.sh/v1
      api_key: os.environ/OPENAI_API_KEY
```

Start the proxy:

```bash theme={null}
export OPENAI_API_KEY="sk-your-tokenlab-key"
litellm --config litellm-config.yaml --port 4000
```

## Call LiteLLM Through OpenAI SDK

```python theme={null}
from openai import OpenAI

client = OpenAI(
    api_key="anything",
    base_url="http://127.0.0.1:4000"
)

response = client.chat.completions.create(
    model="tokenlab-gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

## Direct Python Usage

If you are using LiteLLM as a Python library instead of the proxy, keep the same TokenLab base URL:

```python theme={null}
import litellm

response = litellm.completion(
    model="custom_openai/gpt-5.4",
    api_base="https://api.tokenlab.sh/v1",
    api_key="sk-your-tokenlab-key",
    messages=[{"role": "user", "content": "Summarize this repo."}]
)
```

## Best Practices

<AccordionGroup>
  <Accordion title="Prefer custom_openai for TokenLab">
    Treat TokenLab as an OpenAI-compatible endpoint unless you have a very specific reason to build a more complex provider mapping.
  </Accordion>

  <Accordion title="Use LiteLLM when you need one more gateway layer">
    LiteLLM makes sense when your own platform wants virtual keys, extra model-selection policy, or centralized logs in front of TokenLab.
  </Accordion>

  <Accordion title="Keep native-provider expectations realistic">
    OpenAI-compatible translation layers are great for broad compatibility, but they are not the right place to promise every provider-native feature.
  </Accordion>
</AccordionGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Connection errors">
    * Verify `api_base` is exactly `https://api.tokenlab.sh/v1`
    * Make sure LiteLLM can reach TokenLab over the public internet
    * If you run the proxy locally, verify the OpenAI client points to your LiteLLM port instead of TokenLab directly
  </Accordion>

  <Accordion title="Authentication errors">
    * Check that LiteLLM is reading the right `OPENAI_API_KEY`
    * Confirm the TokenLab key starts with `sk-`
    * Confirm the key is active in TokenLab dashboard
  </Accordion>

  <Accordion title="Model not found">
    * Verify the TokenLab model name in `custom_openai/<model>`
    * Keep your LiteLLM `model_name` alias separate from the real TokenLab model id
  </Accordion>
</AccordionGroup>