One-line Install

If you have Docker, this is the fastest way to get llms.py running with a chat UI, an OpenAI-compatible API, and the providers you have keys for:

curl -fsSL https://llmspy.org/install.sh | bash

It pulls the latest image, installs an llms command on your PATH, and opens a setup screen to pick providers and enter API keys. When it finishes:

llms up                    # start the server on http://localhost:8000
llms ls                    # list enabled providers and models
llms "what is 2+2?"        # ask the default model

Nothing is installed outside ~/.llms and your local bin directory, and nothing needs sudo.

Prerequisites

Docker, and the Docker daemon actually running. That's it — Python, bun, the .NET SDK and typst that llms.py extensions use are all inside the image.

If Docker isn't installed the script stops and prints the right command for your platform:

brew install --cask docker              # macOS
curl -fsSL https://get.docker.com | sh  # Linux

Choosing providers

The installer opens a picker listing every provider in providers.json:

 llms.py — providers

 ●  Anthropic            ANTHROPIC_API_KEY          saved  sk-a..7Yq2      13
 ○  Cerebras             CEREBRAS_API_KEY           -      -                3
 ●  Groq                 GROQ_API_KEY               shell  gsk_..1f9c      15
 ○  Ollama               -                          local  -                0
 ○  OpenAI               OPENAI_API_KEY             -      -               47

 ↑↓ move  space toggle  enter set key  x clear  a all  n none  s save  q quit

The source column shows where each key came from:

So if you already have GROQ_API_KEY or OPENAI_API_KEY exported in your shell, they're picked up automatically and setup is a single keystroke: s.

GITHUB_TOKEN is deliberately never pre-selected — it's usually the gh CLI's token rather than a Copilot subscription, so you have to opt in.

Press enter on any provider to paste an API key. The docs URL for getting one is shown under the list. Input is hidden, and the key is echoed back masked so you can confirm the paste worked.

Saving writes your keys to ~/.llms/.env (mode 600) and enables the corresponding providers in ~/.llms/llms.json.

Re-running it

The script is designed to be re-run. It's the update mechanism:

curl -fsSL https://llmspy.org/install.sh | bash

This pulls the latest image, tells you whether anything changed, refreshes the llms command, and re-opens the picker — now showing which providers you've already configured, so you can add another key or turn one off.

To skip the picker, or only show the picker:

curl -fsSL https://llmspy.org/install.sh | bash -s -- --no-setup
llms setup                                                        # picker only
llms update                                                       # image only

Options

Pass options after | bash -s --:

curl -fsSL https://llmspy.org/install.sh | bash -s -- --port 3000 --no-setup

The llms command

The wrapper passes llms CLI arguments straight through to the container, so every example in these docs works unchanged:

llms ls                              # list enabled providers and models
llms ls anthropic                    # filter to one provider
llms --check groq                    # verify a provider's models
llms -m gpt-5 "explain monads"       # pick a model
llms --image ./photo.png "describe"  # the current directory is mounted at /work

Plus container management:

llms --serve [port] is accepted as an alias for llms up.

The server starts with --restart unless-stopped, so it comes back after a reboot. llms down stops that.

If you already have llms-py installed with pip

If a different llms is already on your PATH, the installer says so and additionally installs the Docker version as llms-docker, leaving your existing llms untouched. Both read the same ~/.llms config directory.

Logging

The server is quiet by default. To see requests:

llms up --verbose          # request logging
llms up --debug            # verbose + debug logging
llms up --debug -f         # ...and follow the logs
llms restart --debug       # turn it on for an already-running server
llms --debug ls            # one-shot commands too
llms logs                  # last 200 lines
llms logs -f               # follow
llms status                # shows the active log level

These set VERBOSE=1 / DEBUG=1 inside the container, which is what llms.py itself reads — the equivalent of DEBUG=1 llms --serve 8000 --verbose. Make it permanent with LLMS_VERBOSE=1 or LLMS_DEBUG=1 in ~/.llms/config.

Voice input

The microphone is in your browser, so nothing needs to be passed into the container. Add a speech-to-text key and restart:

echo 'GROQ_API_KEY=gsk_...' >> ~/.llms/.env
llms restart

Choose the provider and model under defaults in ~/.llms/llms.json:

{
  "defaults": {
    "voice": {
      "provider": "groq",
      "model": "whisper-large-v3"
    }
  }
}

See Voice Input for local speech-to-text servers, the other modes, and every setting.

If the microphone button doesn't appear, it's almost certainly the browser, not llms.py: getUserMedia only exists over HTTPS or on localhost/127.0.0.1. The default llms up binds to 127.0.0.1 so it works, but browsing to http://<lan-ip>:8000 silently disables the microphone API. Use an SSH tunnel or a TLS reverse proxy.

Your own environment variables

Everything in ~/.llms/.env is passed into the container, not just API keys:

echo 'TZ=Australia/Perth' >> ~/.llms/.env
llms restart

llms setup preserves anything there that isn't a provider API key.

What it creates

Every setting can be overridden per-command or edited in ~/.llms/config:

LLMS_PORT=3000 llms up
LLMS_BIND=0.0.0.0 llms up                            # expose on your LAN
LLMS_IMAGE=ghcr.io/servicestack/llms:4.0.10 llms ls  # pin a version

Uninstalling

llms uninstall            # removes the command and container
rm -rf ~/.llms            # also removes config, keys and chat history

Prefer not to pipe to bash?

Download it first and read it — it's a single self-contained shell script:

curl -fsSL https://llmspy.org/install.sh -o install.sh
less install.sh
bash install.sh

Or skip it entirely and use Docker directly, or install with pip.