Chapter 06: Self-Hosting an Open Model as a Server

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="sk-not-needed")

with client.audio.speech.with_streaming_response.create(
    model="tts-1", voice="cs_CZ-jirka-medium",
    input="Dobrý den, vítejte.",
) as resp:
    resp.stream_to_file("out.mp3")

The trade-off: OpenAI's schema has no first-class field for language or voice cloning reference audio. Self-host projects handle this by overloading the voice field (e.g. voice="cs_CZ-jirka-medium") or by adding non-standard extra endpoints. So target OpenAI-compatible for the 90% path, but expect to drop to a project's native API for cloning or fine language control.

Pattern 1 — Roll your own FastAPI wrapper

If you only need one model and one language pair, a ~40-line FastAPI app is the most transparent option. Here is a minimal Piper wrapper that is genuinely production-shaped (streaming response, voice selection, caching hook):

# app.py
import io, hashlib, subprocess, wave
from fastapi import FastAPI
from fastapi.responses import Response
from pydantic import BaseModel

app = FastAPI()
VOICES = {  # OpenAI-style alias -> piper model path
    "alloy":  "/voices/en_US-amy-medium.onnx",
    "jirka":  "/voices/cs_CZ-jirka-medium.onnx",
}

class SpeechReq(BaseModel):
    model: str = "tts-1"
    input: str
    voice: str = "alloy"
    response_format: str = "wav"

def synth(text: str, model_path: str) -> bytes:
    # piper reads text on stdin, writes WAV to stdout
    p = subprocess.run(
        ["piper", "--model", model_path, "--output_file", "-"],
        input=text.encode("utf-8"), capture_output=True, check=True,
    )
    return p.stdout

@app.post("/v1/audio/speech")
def speech(req: SpeechReq):
    model_path = VOICES.get(req.voice, VOICES["alloy"])
    audio = synth(req.input, model_path)          # add a cache lookup here
    return Response(content=audio, media_type="audio/wav")

This is enough for a single-user app or an internal tool. What it lacks (and why ready-made projects exist): proper async concurrency, format conversion to mp3/opus, request queuing under load, multiple engines, and streaming chunked output. Don't build all of that yourself unless you have a reason to.

Pattern 2 — Use a ready-made OpenAI-compatible server

This is the recommended path for most readers. Several mature open-source projects already wrap the popular models behind the OpenAI schema, ship Docker images, and handle concurrency and format conversion. Here is how they compare for the English+Czech, free-or-cheap brief.

A few honest notes on this table:

• Czech is the hard filter, and it splits the field cleanly. Anything Piper-backed gets Czech essentially for free because Piper ships a real Czech voice (cs_CZ-jirka-medium, trained from a Czech speaker, available in the rhasspy/piper-voices Hugging Face repo). Kokoro has no Czech, full stop — its language list as of early 2026 is English (US/UK), Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese, and Mandarin. So a Kokoro-only server cannot be your Czech path. If you want Kokoro's nicer English and Czech, run two engines (Kokoro for en, Piper for cs) — openedai-speech and speaches both let you do this in one container.
• XTTS-v2 lists Czech among its 17 languages and can voice-clone, but in practice Czech XTTS output is noticeably weaker than its English/Spanish, with prosody and the occasional mispronounced word. Treat XTTS-Czech as "usable, sometimes good, verify on your own text" — not a guaranteed win over Piper, and it needs a GPU.
• openai-edge-tts is a cheat code with an asterisk. It relays to Microsoft Edge's online TTS (the same neural voices as Edge's "Read Aloud"), which includes excellent Czech (cs-CZ-AntoninNeural, cs-CZ-VlastaNeural). It is free and trivially light. But it is not offline or self-contained — it depends on an undocumented Microsoft endpoint that can change or rate-limit, and it's a gray-area use of a consumer service. Great for a hobby project; risky as a product dependency.

Pattern 3 — Wyoming protocol (only if you're in the Home Assistant world)

wyoming-piper exposes Piper over the Wyoming protocol (a simple TCP protocol used by the Rhasspy/Home Assistant voice ecosystem) rather than HTTP. Unless you are integrating with Home Assistant or another Wyoming consumer, skip it — for an app, you want HTTP/OpenAI-compatible. Mentioned here only so you don't accidentally pick the Wyoming image when you wanted the HTTP one.

CPU vs GPU: what each model actually needs

This is the single biggest cost lever. Match the model to the hardware you can afford.

* RTF = seconds of compute per second of audio; lower is better, < 1.0 means faster than real-time. These are approximate community/maintainer figures, not lab benchmarks — verify on your own hardware and text.

Practical reading of this table for the brief:

• If "free" dominates: Piper on a cheap CPU VPS. No GPU rental, runs faster than real-time, has Czech. This is the floor and it's a very good floor.
• If you want clearly more natural English and can spend a little: Kokoro on a small GPU (or even Kokoro-ONNX on CPU if latency tolerance is loose), with Piper alongside for Czech.
• Voice cloning your own narrator's voice in both languages: XTTS-v2 on a GPU, accepting weaker Czech.

Concurrency, batching, and queuing

A naive wrapper that runs the model synchronously will serialize every request — fine for one user, a disaster for ten. Three things to get right:

1. Don't block the event loop. Model inference is CPU/GPU-bound and synchronous. In FastAPI, run it in a thread/process pool (run_in_threadpool, or a dedicated worker process) so the async server stays responsive. The ready-made projects already do this.
2. Bound the concurrency. A GPU has finite VRAM; ten simultaneous XTTS jobs will OOM. Put a semaphore or a small request queue in front of the model (e.g. allow N concurrent inferences, queue the rest). Piper on CPU scales by process count — run several Piper workers and load-balance, since each Piper invocation is single-threaded-ish.
3. Scale horizontally with replicas, not threads. For Piper especially, the clean pattern is multiple identical containers behind a load balancer (or docker compose --scale). Each is cheap.

Batching (combining multiple requests into one GPU forward pass) helps throughput for high-volume Kokoro/XTTS deployments but adds latency and complexity; for "reading my texts" workloads it's usually unnecessary. The more impactful trick is chunking long input — split the text into sentences/paragraphs, synthesize them in parallel (or stream them), and stitch. Kokoro-FastAPI does this auto-stitching for you.

Streaming audio over HTTP

For "read this paragraph aloud," users perceive latency as time-to-first-audio, not total time. Streaming lets playback start while the rest is still synthesizing.

• Chunked transfer / StreamingResponse: the server flushes audio bytes as they're produced. For raw pcm or wav this is straightforward; the client plays bytes as they arrive. Kokoro-FastAPI exposes a dedicated streaming endpoint and supports pcm/opus for low-latency playback; openedai-speech streams as well.
• Sentence-level pseudo-streaming: even without true intra-utterance streaming, you can split input into sentences and emit each as a separate audio chunk. Simple, robust, works with any engine including Piper.
• Client side: in a browser, the Media Source Extensions API or a streaming <audio> source consumes chunked audio; in a backend pipeline, just write the stream to a file or pipe it onward.

FastAPI streaming sketch:

from fastapi.responses import StreamingResponse

@app.post("/v1/audio/speech")
def speech(req: SpeechReq):
    def gen():
        for sentence in split_sentences(req.input):
            yield synth(sentence, VOICES[req.voice])   # bytes per sentence
    return StreamingResponse(gen(), media_type="audio/wav")

Caching — the cheapest performance win

If your app reads the same texts repeatedly (UI strings, article intros, notification phrases), cache the audio. A cache key is hash(text + voice + format + speed); the value is the audio blob.

• In-process LRU for a single replica (fast, lost on restart).
• Disk or object storage (/cache/<sha256>.mp3) for persistence across restarts; serve cache hits with zero inference.
• Redis if you have multiple replicas and want a shared cache.

import hashlib, os
def cache_key(text, voice, fmt, speed):
    raw = f"{text}|{voice}|{fmt}|{speed}".encode()
    return hashlib.sha256(raw).hexdigest()
# on hit: return open(f"/cache/{key}.{fmt}", "rb").read()

For a "read my texts" product where content is often static or semi-static, caching can turn 90%+ of requests into instant file serves, which collapses your GPU/CPU bill.

Where to host, and what it costs (2026, approximate)

How to read the serverless math: Kokoro generates ~3–10× faster than real-time on a 4090-class GPU, so one minute of audio might be ~6–20 seconds of billed GPU time. At ~$0.0004/sec that's roughly $0.0024–$0.008 per minute of audio in raw compute — but you must add cold-start time (loading the model into VRAM, often 10–40s on a cold container), which on serverless you may or may not be billed for depending on platform. With scale-to-zero and bursty usage, serverless GPU often beats a 24/7 GPU box. With steady high volume, the dedicated box wins.

The honest recommendation for "free or near-free": start with Piper on a CPU VPS (or your existing server). It is genuinely free-to-operate at small scale, covers both Czech and English with real voices, and needs no GPU. Add Kokoro (CPU ONNX, or a serverless-GPU function called only for English) only if you decide Piper's English isn't natural enough for you. Reserve dedicated GPU and XTTS for when you specifically need voice cloning at volume.

A concrete deployment: openedai-speech via docker-compose

This gives you one OpenAI-compatible endpoint that serves Czech via Piper and English via Piper or XTTS, with a persistent cache.

# docker-compose.yml
services:
  tts:
    image: ghcr.io/matatonic/openedai-speech-min:latest   # CPU/Piper image; use the CUDA image for XTTS
    container_name: openedai-speech
    ports:
      - "8000:8000"
    volumes:
      - ./voices:/app/voices            # Piper .onnx voices (incl. cs_CZ-jirka-medium)
      - ./config:/app/config            # voice_to_speaker.yaml mapping OpenAI names -> models
      - ./cache:/app/cache              # persistent audio cache
    environment:
      - TTS_HOME=/app/voices
    restart: unless-stopped
    # For XTTS/Kokoro on GPU, add:
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - capabilities: [gpu]

Pull the Czech voice once (Piper voices live in the rhasspy/piper-voices HF repo):

mkdir -p voices && cd voices
# cs_CZ-jirka-medium: .onnx model + .onnx.json config
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/cs/cs_CZ/jirka/medium/cs_CZ-jirka-medium.onnx
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/cs/cs_CZ/jirka/medium/cs_CZ-jirka-medium.onnx.json

Map an OpenAI voice name to it in config/voice_to_speaker.yaml, then bring it up:

docker compose up -d

Sample request (works with curl, your backend, or the OpenAI SDK):

curl -s http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-anything" \
  -d '{
        "model": "tts-1",
        "input": "Dobrý den, toto je ukázka české syntézy řeči.",
        "voice": "jirka",
        "response_format": "mp3"
      }' \
  --output cz.mp3

Alternative: Kokoro-FastAPI for best-effort English

If you want Kokoro's more natural English and don't mind running it separately, its images are CPU-or-GPU ready:

# CPU (ONNX)
docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest
# NVIDIA GPU (PyTorch)
docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest

curl -s http://localhost:8880/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"kokoro","input":"This is a Kokoro voice test.","voice":"af_bella","response_format":"mp3"}' \
  --output en.mp3

Then route by language in your app: cs → openedai-speech/Piper on :8000, en → Kokoro on :8880. Two small containers, one natural-sounding bilingual TTS service, near-zero cost on CPU.

A minimal custom Dockerfile (if you build your own wrapper)

FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
      ffmpeg && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir piper-tts fastapi uvicorn
COPY app.py /app/app.py
WORKDIR /app
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "2"]

Production hardening checklist

• Auth: even an "open" service needs an API key/header so the public internet can't run free synthesis on your box. Put it behind a reverse proxy (Caddy/nginx/Traefik) with TLS.
• Rate limiting & max input length: reject 100k-character payloads; cap per-request length and per-IP rate.
• Health checks: add /health so your orchestrator/load balancer can restart dead replicas.
• Observability: log request length, latency, cache hit/miss; watch p95 time-to-first-byte.
• Voice/model pinning: pin image tags and voice file versions so a rebuild doesn't silently change how your audio sounds.

Key Takeaways

• Target the OpenAI /v1/audio/speech schema. It's the de-facto standard; building your client against it lets you swap engines (Piper ↔ Kokoro ↔ cloud) by changing a base URL.
• Use a ready-made server, not a from-scratch wrapper, unless you have a reason. openedai-speech (Piper + XTTS), Kokoro-FastAPI (Kokoro), and speaches (Kokoro/Piper + Whisper STT) already handle concurrency, format conversion, and Docker.
• Czech splits the field cleanly: Piper has a real Czech voice (cs_CZ-jirka-medium); Kokoro has no Czech at all. For bilingual EN+CZ, the clean pattern is Kokoro (or Piper) for English + Piper for Czech, possibly in one container.
• Hardware: Piper = CPU and basically free; Kokoro = small GPU for a big speed win (or slow CPU ONNX); XTTS = GPU-required and only worth it for voice cloning (with weaker Czech).
• Get concurrency right: don't block the event loop, bound GPU concurrency with a queue/semaphore, and scale Piper by running multiple replicas.
• Stream for perceived latency and cache aggressively — for "read my texts," caching static phrases can eliminate most inference cost.
• Hosting: CPU VPS (~€5–30/mo) for Piper is the free-or-near-free sweet spot; serverless GPU (RunPod/Modal/Replicate, $0.003–0.01/min of audio + cold starts) beats a 24/7 GPU box for bursty traffic; a dedicated GPU box (€180+/mo) only pays off at steady volume.

Key References

• matatonic. GitHub — matatonic/openedai-speech. 2024–2026. https://github.com/matatonic/openedai-speech — OpenAI-compatible TTS server wrapping Piper and Coqui XTTS-v2; the primary drop-in self-host option with Czech via Piper.
• remsky. GitHub — remsky/Kokoro-FastAPI. 2025–2026. https://github.com/remsky/Kokoro-FastAPI — Dockerized FastAPI wrapper for Kokoro-82M with CPU ONNX and NVIDIA GPU PyTorch images, streaming endpoint, and OpenAI compatibility (English-focused, no Czech).
• speaches-ai. GitHub — speaches-ai/speaches / speaches.ai. 2025–2026. https://github.com/speaches-ai/speaches — OpenAI-compatible STT and TTS server supporting Kokoro, Piper, and faster-whisper in one service.
• rhasspy. GitHub — rhasspy/piper. 2023–2026. https://github.com/rhasspy/piper — Fast local neural TTS; runs faster than real-time on CPU (incl. Raspberry Pi 4) and ships a Czech voice.
• rhasspy. Hugging Face — rhasspy/piper-voices. 2024–2026. https://huggingface.co/rhasspy/piper-voices — Voice model repository including cs_CZ-jirka-medium, the Czech Piper voice.
• rhasspy. GitHub — rhasspy/wyoming-piper. 2023–2026. https://github.com/rhasspy/wyoming-piper — Wyoming-protocol Piper server for Home Assistant voice stacks (non-HTTP).
• travisvn. GitHub — travisvn/openai-edge-tts. 2024–2026. https://github.com/travisvn/openai-edge-tts — Lightweight OpenAI-compatible endpoint relaying to Microsoft Edge neural voices (good Czech, but cloud-dependent / gray-area).
• OpenAI. Audio / Text-to-speech — OpenAI API docs. 2024–2026. https://platform.openai.com/docs/guides/text-to-speech — Canonical /v1/audio/speech request schema (model, input, voice, response_format, speed) that self-host projects imitate.
• RunPod. Serverless GPU Pricing / Pricing. 2025–2026. https://www.runpod.io/pricing — Per-second serverless GPU rates used for the cost estimates (RTX-4090-class and A100-80GB tiers).
• Hetzner. Dedicated GPU servers (GEX matrix) & Cloud pricing. 2025–2026. https://www.hetzner.com/dedicated-rootserver/matrix-gpu/ and https://www.hetzner.com/cloud/ — Reference for CPU VPS (€5–30/mo) and small dedicated GPU (€180+/mo) hosting costs.
• VirtualizationHowto. Self-Hosted AI Voices with Kokoro-FastAPI. 2025. https://www.virtualizationhowto.com/2025/07/self-hosted-ai-voices-with-kokoro-fastapi-free-fast-and-local-tts-for-home-assistant/ — Practical walkthrough confirming Kokoro-FastAPI Docker deployment and performance characteristics.