omni-vox

OpenAI-compatible inference gateway.

Endpoints

• GET /llm/v1/models
• POST /llm/v1/chat/completions — supports stream: true
• POST /llm/v1/completions
• GET /llm/docs — interactive API reference

Quick start

curl -H "Authorization: Bearer $LLM_API_KEY" \
     -H 'Content-Type: application/json' \
     "$HOST/llm/v1/chat/completions" \
     -d '{"model":"gemma-4-12b",
          "messages":[{"role":"user","content":"Say hello."}]}'

Python

from openai import OpenAI

client = OpenAI(base_url="$HOST/llm/v1", api_key="...")
client.chat.completions.create(
    model="gemma-4-12b",
    messages=[{"role": "user", "content": "Say hello."}],
)