Ragmux

Self-hosted AI gateway

Every model behind one API.Every answer grounded in your documents.

Ragmux is a single Go binary that sits between your applications and OpenAI, Anthropic, Gemini, DeepSeek, Ollama or your own vLLM — speaking one OpenAI-compatible API, and injecting retrieval from your own PDFs into every request it forwards.

Get startedRead the docs

Docker Compose/PostgreSQL 17 + pgvector/AGPL-3.0-or-later

~/ragmuxbash
$ cp .env.example .env
$ echo "SECRET_KEY=$(openssl rand -hex 32)" >> .env
$ docker compose up -d
✔ragmuxstarted:8080
✔pgvector/pgvector:pg17healthy:5432
Initial admin password printed once — docker logs ragmux
→ http://localhost:8080/admin/
Routes to
  • OpenAI
  • Anthropic
  • Gemini
  • DeepSeek
  • Ollama
  • vLLM · LM Studio · LiteLLM
  • Static Go binary, CGO_ENABLED=0, distroless base
  • Two containers — no Redis, no separate vector database
  • Provider keys AES-256-GCM encrypted at rest

How it works

One request in. The right model, with the right context, out.

A project key is the whole configuration surface. It decides which provider answers, which system prompt applies, which documents are searched and what the caller is allowed to spend.

Your app

POST /v1/chat/completions
Authorization: Bearer sk-proj-…
{"model": "default",
 "stream": true,
 "messages": [ … ]}

The official OpenAI SDKs work unchanged. The model field is echoed back — the project decides the real one.

Ragmux:8080

  1. 01Resolve the projectKey → model connection, system prompt, rate limits and token budget.
  2. 02Retrieve, if the project has a storeEmbed the query, hybrid search the chunks, inject the top-k passages into the system prompt.
  3. 03Adapt, forward, normaliseTranslate to the provider's schema — including Anthropic tool calls — and stream the answer back as OpenAI SSE.

Upstream

  • OpenAI
  • Anthropic
  • Gemini
  • DeepSeek
  • Ollama /api/chat
  • custom_openai

Credentials stay server-side and are never returned by the API.

PostgreSQL + pgvector

  • users & roles
  • connections
  • projects
  • documents (bytea)
  • chunks
  • HNSW vector indexes
  • request logs

Drop-in

Change two lines. Keep your SDK.

The gateway answers /v1/chat/completions and /v1/models in the OpenAI schema, JSON and SSE alike. Nothing else in your codebase moves.

  • Anthropic and Gemini requests and streams are translated for you — tool calls included for Anthropic.
  • Every response carries x-ragmux-rag-hits — how many passages were injected.
  • Over a limit you get an OpenAI-style 429 with Retry-After and x-ratelimit-* headers.
app/client.pydiff
from openai import OpenAI
client = OpenAI(
-    base_url="https://api.openai.com/v1",
-    api_key="sk-…",
+    base_url="https://gateway.internal/v1",
+    api_key="sk-proj-…",
)
resp = client.chat.completions.create(
    model="default", stream=True,
    messages=[{"role": "user",
             "content": "How many vacation days do I get?"}],
)
# the handbook PDF is searched and injected server-side

Retrieval

Your documents belong in the request — not in a second system.

Upload a file to a RAG store and an ingestion worker takes it from pending to ready. The vectors land in the same database as everything else.

  1. 01ParsePDF, DOCX, HTML, Markdown and plain text, straight into the database as bytea.
  2. 02ChunkSection-aware splitting, with contextual chunks so a passage still makes sense out of its page.
  3. 03EmbedThrough the embedding connection you chose for the store; indexed with HNSW in pgvector.
  4. 04SearchVector similarity and Postgres full-text, fused with reciprocal rank fusion.
  5. 05InjectTop-k passages prepended to the system prompt, counted in the response headers.

Optional LLM reranking

Let a model re-order the candidate passages before they reach the prompt, when precision matters more than latency.

A distance threshold

max_distance drops weak matches instead of padding the prompt with noise.

Per-store text search config

fts_config picks the Postgres dictionary, so the lexical half of the search speaks your language.

Control plane

Keys, roles, limits — and a log of everything that touched them.

What a shared provider key can't give you: who spent what, on which project, under which prompt.

Projects and keys

One sk-proj-… key per project, mapped to a model connection, a system prompt and an optional RAG store. Provider credentials are AES-256-GCM encrypted at rest.

Roles and membership

admin, editor, viewer, plus per-project membership. Non-admins only see their own projects, and a foreign project id answers 404 rather than 403.

Rate limits and budgets

Requests and tokens per minute, plus daily and monthly token budgets in UTC windows — counted in the database, so replicas share one ceiling.

Login protection

Per-username and per-IP attempt limits with a lockout window, stored in Postgres. The client IP is the TCP peer unless you opt into proxy headers.

Audit log

Every login and every management action is recorded, readable at /admin/api/audit and in its own dashboard tab.

Metrics and retention

Tokens, latency, status, streaming and RAG use per request; summaries and daily series on top, with a retention window you set and an hourly janitor that enforces it.

Dashboard & API

The UI is just another client.

The dashboard at /admin/ ships inside the binary and manages models, RAG stores, documents, projects, metrics, users and the audit log — with a playground to try a project key before you wire it up. Everything it does is a REST call you can make yourself.

No build step

Vanilla JS, embedded

Same auth

Bearer session token

The surface
  • POST/v1/chat/completionsthe proxy — JSON or SSE
  • GET/v1/modelswhat this key may call
  • POST/admin/api/modelsadd a provider connection
  • POST/admin/api/rag-storescreate a store
  • POST/admin/api/rag-stores/{id}/documentsupload a file
  • POST/admin/api/projectsthe key is shown once
  • GET/admin/api/auditwho changed what
  • GET/admin/api/systemversions, database size, backup
  • GET/healthzfor your orchestrator

Quick start

From nothing to a grounded answer.

  1. 01Bring up the two containers

    $ cp .env.example .env
    $ echo "SECRET_KEY=$(openssl rand -hex 32)" >> .env
    $ docker compose up -d

  2. 02Add a model connection and a RAG store

    $ curl -s localhost:8080/admin/api/models -H "$AUTH" \
      -d '{"name":"claude","provider_type":"anthropic", …}'
    $ curl -s localhost:8080/admin/api/rag-stores/1/documents \
      -H "$AUTH" -F file=@handbook.pdf

  3. 03Create the project and call it like OpenAI

    $ curl -N localhost:8080/v1/chat/completions \
      -H "Authorization: Bearer sk-proj-…" \
      -d '{"model":"default","stream":true,"messages":[…]}'
    x-ragmux-rag-hits: 4

Documentation

Written for the person on call.

Browse the repository

Configuration

Every environment variable and default, the -healthcheck and -version flags, reverse-proxy notes.

API reference

Authentication, the role required per endpoint, request and response examples, the error envelope.

Retrieval

Formats, contextual chunking, hybrid search and fts_config, reranking, the context format.

Users, roles and limits

The roles matrix, project membership, login protection, the audit log, budgets and retention.

Providers

Provider types and endpoints, the Anthropic, Gemini and Ollama translation details, passthrough and model echo.

Backup and restore

What to back up, the scripts, scheduled backups, point-in-time recovery and the restore runbook.

Not in 0.2.0Gemini tool calling · OCR for scanned PDFs · a Prometheus endpoint · SSO and OIDC login · prompt caching passthrough · an import tool for 0.1 SQLite databases

Your keys, your documents, your database.

Ragmux is free software under the AGPL-3.0-or-later. Run it on a laptop, run it in your cluster — nothing phones home.

Get startedGitHub
docker compose up -d

Arrow keys to move, Enter to open.