Installation

Using pip

The easiest way to install llms.py is using pip:

pip install llms-py

Using Docker

Pre-built Docker Images

Pull and run the latest image:

docker pull ghcr.io/servicestack/llms:latest

docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latest

Using docker compose (Recommended)

Download the docker-compose.yml file:

curl -O https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml

Create a .env file with your API keys:

OPENROUTER_API_KEY=sk-or-...
GROQ_API_KEY=gsk_...
GOOGLE_FREE_API_KEY=AIza...

Create the host data directory before Docker mounts it:

mkdir -p ~/.llms

On Linux, add PUID and PGID to .env using the values from id -u and id -g for the account that owns the data directory. For example:

PUID=1000
PGID=1000

Compose defaults to 1000:1000 when these variables are unset. The selected user must be able to write to the directory. See Docker permissions if the directory already exists with different ownership.

Start the service:

docker compose up -d

Build from Source

If you've cloned the repository:

git clone https://github.com/ServiceStack/llms

cd llms

docker compose -f docker-compose.local.yml up -d --build

Setup

Setup environment variables for the providers you want to use.

Updating

To update to the latest version:

pip install llms-py --upgrade

After upgrading, it's recommended to also upgrade any external extensions:

llms --update all

For Docker:

docker pull ghcr.io/servicestack/llms:latest

Reset to latest configuration

New versions sometimes include changes to llms.json config which isn't automatically updated.

Use the --reset option to reset the default configuration files back to its factory defaults. Available reset options (config, providers, all):

config - Reset ~/.llms/llms.json to default

llms --reset config

providers - Reset ~/.llms/providers.json and ~/.llms/providers-extra.json

llms --reset providers

On this page