LLM Lab User Guide

LLM Lab is a unified API gateway for large language models. It provides a single OpenAI-compatible endpoint that routes your requests to the best available model based on your configuration, cost targets, and quality requirements.

1. Introduction

With LLM Lab, you can:

• Access multiple LLM providers through one API
• Route requests intelligently across models
• Monitor usage, costs, and latency in real time
• Manage team API keys and permissions
• Top up a prepaid wallet and control spending

This guide is for both customers (developers and teams using the API) and administrators (operators managing the platform).

2. Quick Start

1. Sign Up — visit the LLM Lab web application, create an account or log in, and confirm your email if required.
2. Create an API Key — navigate to API Keys in the customer portal, click Create API Key, and copy the key immediately (it is shown only once).
3. Send a Request:

curl -X POST https://your-llm-lab-domain/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Hello, world!"}
    ]
  }'

3. Authentication

LLM Lab uses API keys passed in the Authorization header.

Authorization: Bearer YOUR_API_KEY

Security best practices

• Store API keys in environment variables, never in source code
• Rotate keys regularly
• Delete keys that are no longer needed
• Use separate keys for development and production

Admin authentication

• Admins log in through the web interface
• Sessions use secure, HttpOnly, SameSite cookies
• Do not share admin credentials

4. Making Your First Request

Endpoint

POST /v1/chat/completions

Request body

{
  "model": "gpt-4o-mini",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is LLM Lab?"}
  ],
  "temperature": 0.7,
  "max_tokens": 256
}

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1786123456,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "LLM Lab is a unified API gateway for large language models..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 24,
    "total_tokens": 36
  }
}

Streaming

Set stream: true to receive Server-Sent Events (SSE):

{
  "model": "gpt-4o-mini",
  "messages": [{"role": "user", "content": "Hello"}],
  "stream": true
}

5. Supported Models & Providers

LLM Lab supports multiple providers. The exact list depends on what the admin has enabled.

Listing available models

curl -H "Authorization: Bearer YOUR_API_KEY" \
  https://your-llm-lab-domain/v1/models

Selecting a model

Use the model field in your request. You can use either the model alias configured in LLM Lab (e.g., gpt-4o-mini) or a Smart Model name if routing rules are configured.

6. Smart Model Routing

Smart Models let you define routing rules so requests are sent to the most appropriate model automatically.

Why use smart routing?

• Reduce costs by sending simple queries to cheaper models
• Improve quality by routing complex tasks to stronger models
• Add fallback providers for reliability

How it works

1. An admin creates a Smart Model (e.g., smart-coding-assistant)
2. Rules are defined with criteria such as cost target, latency target, required capabilities, and content patterns
3. Customers use the Smart Model name in their API requests
4. LLM Lab selects the best matching enabled model for each request

Example request with a Smart Model

{
  "model": "smart-coding-assistant",
  "messages": [
    {"role": "user", "content": "Write a Python function to reverse a string."}
  ]
}

7. Usage Dashboard

Customer dashboard

• Total requests and tokens
• Daily cost trend
• Requests over time
• Token usage over time
• Average latency trend
• Success vs. failed requests

Admin dashboard

• Platform-wide totals
• Today's revenue
• Active users and models
• Top users and top models
• Recent activity

8. Billing & Wallet

LLM Lab uses a prepaid wallet model.

Topping up

1. Go to Billing in the customer portal
2. Choose a top-up amount
3. Complete payment through Stripe
4. Your wallet balance updates immediately

Balance checks

• Each request checks your wallet balance before it is sent
• If your balance is insufficient, the API returns a 402 Payment Required response

Invoices

All top-ups and usage charges appear in the Invoices section. Download receipts for accounting purposes.

Admin billing controls

Admins can configure Stripe keys and payment settings, view all customer invoices, manage customer payment methods, and set platform-wide pricing margins.

9. API Key Management

Creating a key

1. Go to API Keys
2. Click Create API Key
3. Enter a descriptive name (e.g., "Production Web App")
4. Copy the key value immediately

Revoking a key

1. Find the key in the list
2. Click Delete or Revoke
3. The key becomes invalid immediately

Best practices

• Use one key per application or environment
• Rotate keys every 90 days
• Monitor usage per key in the dashboard

10. Sandbox

The Sandbox is a built-in chat interface for testing models without writing code.

1. Go to Sandbox
2. Select a model from the dropdown
3. Type a message and send
4. View the model's response

Use it to test prompt behavior, compare model outputs, and validate integrations before deployment.

11. Error Handling

Error response format

{
  "error": {
    "message": "Insufficient balance. Please top up your wallet.",
    "type": "insufficient_balance"
  }
}

Tips

• Always check error.type before retrying
• Implement exponential backoff for 429 and 503 errors
• Log request IDs when contacting support

12. Code Examples

Python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://your-llm-lab-domain/v1",
    api_key=os.environ["LLM_LAB_API_KEY"]
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Node.js

import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: 'https://your-llm-lab-domain/v1',
  apiKey: process.env.LLM_LAB_API_KEY,
});

const completion = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Hello!' }],
});

console.log(completion.choices[0].message.content);

cURL

curl -X POST https://your-llm-lab-domain/v1/chat/completions \
  -H "Authorization: Bearer $LLM_LAB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

13. Admin Guide

Provider management

1. Go to Admin > Providers
2. Add a new provider with display name, adapter type, base URL, and encrypted API key
3. Toggle providers on or off
4. Run health checks from the model list

Model management

1. Go to Admin > Models
2. Add or edit models
3. Map each model alias to an upstream provider model ID
4. Set pricing (input cost per 1M tokens, output cost per 1M tokens)
5. Enable or disable models for customers

Smart Model setup

1. Go to Admin > Smart Models
2. Create a Smart Model
3. Add routing rules with priorities
4. Define criteria such as cost, latency, or capabilities
5. Save and test in the sandbox

User management

1. Go to Admin > Users
2. Create customer or admin accounts
3. Reset passwords if needed
4. Monitor per-user usage

Payment settings

1. Go to Admin > Payment Settings
2. Configure Stripe publishable and secret keys
3. Set minimum top-up amount
4. Enable or disable Stripe top-ups

14. Troubleshooting

• API key is invalid — verify you copied the full key, check that the key has not been revoked, and ensure the Authorization header uses Bearer prefix.
• Model returns "not found" — confirm the model alias is enabled, check the provider is enabled, and try the exact alias shown in /v1/models.
• Insufficient balance — top up your wallet in the Billing section and check your current balance on the dashboard.
• Requests are slow — check the latency chart in the dashboard, try a different provider or model, and review Smart Model rules for latency targets.
• Provider health check fails — verify the provider API key is correct, check the provider base URL, and confirm the upstream model ID exists at the provider.

15. FAQ

Q: Is LLM Lab a replacement for OpenAI?
A: No. LLM Lab routes requests to upstream providers such as OpenAI, Anthropic, and Kimi. You still pay for upstream usage, plus the LLM Lab margin.

Q: Can I use my existing OpenAI SDK code?
A: Yes. Point the base URL to your LLM Lab domain and use your LLM Lab API key.

Q: How is pricing calculated?
A: Pricing is based on token usage and the configured per-1M-token rates for each model. A platform margin may be added.

Q: What happens if a provider is down?
A: If Smart Model fallback rules are configured, LLM Lab will route the request to an alternative provider.

Q: Can I add custom providers?
A: Yes. Admins can add custom providers that support OpenAI-compatible or Anthropic-compatible APIs.

Q: Is my data secure?
A: Provider API keys are encrypted at rest. Authentication uses secure cookies. JWTs are never exposed to client-side JavaScript.