Getting Started
Installation
How to install llms.py
Using pip
The easiest way to install llms.py is using pip:
pip install llms-pyUsing Docker
Pre-built Docker Images
Pull and run the latest image:
docker pull ghcr.io/servicestack/llms:latestdocker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latestUsing docker compose (Recommended)
- Download the docker-compose.yml file:
- Create a
.envfile with your API keys:
OPENROUTER_API_KEY=sk-or-...
GROQ_API_KEY=gsk_...
GOOGLE_FREE_API_KEY=AIza...- Start the service:
docker compose up -dBuild from Source
If you've cloned the repository:
git clone https://github.com/ServiceStack/llmscd llmsdocker compose -f docker-compose.local.yml up -d --buildSetting Up API Keys
Set environment variables for the providers you want to use:
export OPENROUTER_API_KEY="sk-or-..."
export GROQ_API_KEY="gsk_..."
export GOOGLE_FREE_API_KEY="AIza..."
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENAI_API_KEY="sk-..."
export GROK_API_KEY="xai-..."
export DASHSCOPE_API_KEY="sk-..." # Qwen
export ZAI_API_KEY="sk-..."
export MISTRAL_API_KEY="..."API Key Reference
| Provider | Environment Variable | Description |
|---|---|---|
| openrouter_free | OPENROUTER_API_KEY | OpenRouter FREE models |
| groq | GROQ_API_KEY | Groq API key |
| google_free | GOOGLE_FREE_API_KEY | Google FREE API key |
| codestral | CODESTRAL_API_KEY | Codestral API key |
| ollama | N/A | No API key required |
| openrouter | OPENROUTER_API_KEY | OpenRouter API key |
GOOGLE_API_KEY | Google API key | |
| anthropic | ANTHROPIC_API_KEY | Anthropic API key |
| openai | OPENAI_API_KEY | OpenAI API key |
| grok | GROK_API_KEY | Grok (X.AI) API key |
| qwen | DASHSCOPE_API_KEY | Qwen (Alibaba) key |
| z.ai | ZAI_API_KEY | Z.ai API key |
| mistral | MISTRAL_API_KEY | Mistral API key |
Verification
After installation, verify it's working:
llms lsUpdating
To update to the latest version:
pip install llms-py --upgradeFor Docker:
docker pull ghcr.io/servicestack/llms:latest