Start
Getting started
A self-hosted AI gateway that exposes one OpenAI-compatible API for several LLM providers and adds retrieval from uploaded documents.
Updated:
A self-hosted AI gateway that exposes one OpenAI-compatible API for several LLM providers and adds retrieval from uploaded documents.
Ragmux sits between your applications and LLM providers. Provider credentials stay on the server, and a request can be augmented with passages retrieved from documents you upload (RAG). All state lives in one PostgreSQL database with the pgvector extension: no Redis and no separate vector database. More at https://ragmux.com.
Features
- Providers:
openai,azure_openai,anthropic,gemini,deepseek,ollamaandcustom_openai(vLLM, LM Studio, LiteLLM or any OpenAI-compatible server), pluscohere_rerankandvoyage_rerank. Presets cover Mistral, xAI and Groq; see Providers. - OpenAI-compatible API:
/v1/chat/completions(JSON and SSE streaming),/v1/models,/v1/embeddings,/v1/retrieveand/v1/rerank. The official SDKs work after changingbase_urlandapi_key; Anthropic and Gemini requests are translated, tool calls included. - Anthropic-compatible API:
/v1/messages. The Anthropic SDKs and Claude Code work after changing the base URL. On an Anthropic connection the request and the answer pass through as they are, beta headers included; on any other connection the request is translated, so one Claude client can be routed to Gemini or an OpenAI-compatible server. SeePOST /v1/messages. - Reasoning: one
reasoning_effortsetting is mapped onto each provider’s thinking controls, thinking blocks travel between turns inragmux_thinking, andresponse_formatworks on Anthropic too; see Reasoning effort. - Agents: an MCP server at
/mcpthat exposes the project’s RAG store as a search tool, per-project tool allow and deny lists, andX-Ragmux-Sessionto follow, and cap, an agent’s run; see MCP endpoint and Tool governance. - Routes, retry and fallback: named routes with prioritised, weighted targets, retries before the first byte and a circuit breaker per connection; see Routing.
- RAG stores: PDF, DOCX, HTML, TXT and Markdown, section-aware chunking, hybrid search (pgvector with Postgres full-text ranking or ParadeDB BM25), reranking, a search language per store, document metadata with filtering at query time, no re-embedding of duplicate uploads, and HNSW-indexed embeddings of up to 4000 dimensions; see Retrieval.
- RAG behaviour per project: where the retrieved context goes in the prompt (so a cached
system prompt stays cacheable), a retrieval policy that skips greetings and tool turns,
and a query strategy that condenses a conversation into one standalone question. A request
can narrow retrieval through a
ragmuxobject and receive its sources, streamed or not. - Projects and API keys: one
sk-proj-key per project, mapped to a model connection, a system prompt and an optional RAG store; scopedsk-user-andsk-mgmt-keys with expiry and revocation. - Access control:
admin,editorandviewerroles, project membership, login lockout and an audit log of every management action. - Limits and cost: per-project rate limits and token budgets shared across replicas, prompt-caching passthrough and an editable price table that estimates request cost.
- Observability and scale: request logs, an authenticated Prometheus
/metricsendpoint, OpenTelemetry traces, and several replicas against one PostgreSQL; see Observability and Scaling.
Quick Start
Requires Docker. One container runs the gateway with its own PostgreSQL 17 and pgvector:
docker run -d --name ragmux \
-p 127.0.0.1:8765:8765 \
-e SECRET_KEY="$(openssl rand -hex 32)" \
-v ragmux-data:/data \
ragmux/ragmux:latest
Open http://localhost:8765/admin/. On a fresh database the dashboard asks you to
create the first administrator (username and a password of at least 12 characters);
that form only works while no user exists. For unattended installs pass
-e ADMIN_USER=admin -e ADMIN_PASSWORD=<ADMIN_PASSWORD> instead and the account is
created on first start (the password is 12 characters at least, 72 bytes at most; on an
empty database a shorter one stops the start with an error).
SECRET_KEYencrypts the provider credentials. Without it the gateway generates/data/ragmux/secret.keyinside the volume, which you then have to back up together with the database. Keep the key with your backups.- The port is published on loopback only. Put a TLS-terminating reverse proxy in front
for network access; see Deployment. Change the mapping to
-p 8765:8765only deliberately. - Local models: provider URLs on private networks (Ollama on the Docker host, a vLLM
service on the same Docker network) are refused by default as an SSRF guard. Add
-e PRIVATE_UPSTREAM_ALLOWLIST=host.docker.internal,ollamato the command above; see Private upstreams.
latest is for trying it out. Pin a version in production; see Which image.
Installation
Which image
Three variants are published from every release:
| Tag | What it is | Choose it when |
|---|---|---|
ragmux/ragmux:latest |
All-in-one: the gateway plus its own PostgreSQL 17 + pgvector in one container (Dockerfile.aio). |
You want one container and no database to operate. The default for most installations. |
ragmux/ragmux:latest-app |
Gateway only, on a distroless base; you point it at your own PostgreSQL with DATABASE_URL (Dockerfile). |
You already run PostgreSQL, or you want several gateway replicas against one database. |
ragmux/ragmux:latest-paradedb |
All-in-one on ParadeDB: its PostgreSQL carries pg_search (BM25) as well as pgvector (Dockerfile.aio.paradedb). |
You want the lexical half of hybrid retrieval answered by BM25. |
The -paradedb image takes the same command and volume layout as the default one. Docker
Hub is the primary registry; the tag matrix, the GHCR mirror and the platforms are under
Published images.
A deployment you care about should name the release it was tested against
(ragmux/ragmux:<version>), or the manifest digest that cosign verify reports. Images
are signed from v0.3.1 onward; see
Verifying the container image.
Docker Compose
Compose works from a clone of the repository and is the better route for your own PostgreSQL, the split layout, ParadeDB, several replicas or the scheduled backup profile:
git clone https://github.com/ragmux/ragmux.git
cd ragmux
cp .env.example .env
echo "SECRET_KEY=$(openssl rand -hex 32)" >> .env
docker compose up -d
That is the same all-in-one image and /data volume as the docker run above, with .env
as the one place configuration lives. Keep the generated SECRET_KEY with your backups.
The other files layer on top:
| File | What it runs |
|---|---|
docker-compose.yml |
the default all-in-one container, plus a backup profile |
docker-compose.split.yml |
the gateway (-app) next to its own pgvector/pgvector:pg17; needs SECRET_KEY, POSTGRES_PASSWORD and RAGMUX_DB_PASSWORD in .env |
docker-compose.paradedb.yml |
the split layout with ParadeDB in place of pgvector, for BM25 lexical retrieval |
docker-compose.scale.yml |
an overlay on the split file: several gateway replicas behind nginx; see Scaling |
docker-compose.otel.yml |
an overlay adding an OpenTelemetry Collector; see Observability |
For a PostgreSQL you already operate, set DATABASE_URL and the default container skips
its embedded server, or run the -app image with DATABASE_URL and SECRET_KEY. See
Deployment layouts.
Usage
The steps below use the management API; the dashboard at /admin/ does the same through
forms. It is a single-page UI embedded in the binary: viewers get read-only screens,
editors the forms, administrators the user and audit screens. Every screen is listed in
Dashboard.
1. Log in and keep the session token:
TOKEN=$(curl -s localhost:8765/admin/api/login -H 'Content-Type: application/json' \
-d '{"username":"admin","password":"<ADMIN_PASSWORD>","bearer":true}' | jq -r .token)
AUTH="Authorization: Bearer $TOKEN"
2. Add model connections (a chat model and, for RAG, an embedding model):
curl -s localhost:8765/admin/api/models -H "$AUTH" -H 'Content-Type: application/json' -d '{
"name": "claude", "provider_type": "anthropic", "api_key": "<ANTHROPIC_API_KEY>", "model_name": "claude-sonnet-4-5"}'
curl -s localhost:8765/admin/api/models -H "$AUTH" -H 'Content-Type: application/json' -d '{
"name": "openai-embed", "provider_type": "openai", "api_key": "<OPENAI_API_KEY>", "model_name": "text-embedding-3-small"}'
3. Create a RAG store and upload documents (optional):
curl -s localhost:8765/admin/api/rag-stores -H "$AUTH" -H 'Content-Type: application/json' -d '{
"name": "handbook", "embedding_connection_id": 2}'
curl -s localhost:8765/admin/api/rag-stores/1/documents -H "$AUTH" -F file=@handbook.pdf
curl -s localhost:8765/admin/api/rag-stores/1/documents -H "$AUTH" # status: pending → processing → ready
4. Create a project and get its sk-proj- key (shown only once):
curl -s localhost:8765/admin/api/projects -H "$AUTH" -H 'Content-Type: application/json' -d '{
"name": "support-bot", "model_connection_id": 1, "rag_store_id": 1,
"system_prompt": "You are the company support assistant."}'
5. Call it like OpenAI. The model field is echoed back but the project decides the
real model (a project with routes uses it to pick one).
Retrieved passages are injected into the system prompt (or in front of the last user
message, by project setting) and counted in the x-ragmux-rag-hits header.
curl -N localhost:8765/v1/chat/completions \
-H "Authorization: Bearer <PROJECT_KEY>" -H 'Content-Type: application/json' \
-d '{"model":"default","stream":true,"messages":[{"role":"user","content":"How many vacation days do I get?"}]}'
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8765/v1", api_key="<PROJECT_KEY>")
resp = client.chat.completions.create(model="default", stream=True,
messages=[{"role": "user", "content": "How many vacation days do I get?"}])
for chunk in resp:
# Guard the array: an upstream may send a frame with no choices, and
# asking for usage adds a final chunk that carries only the token counts.
if chunk.choices:
print(chunk.choices[0].delta.content or "", end="")
The Anthropic SDKs and Claude Code take the same key in x-api-key; point them at the
gateway’s base URL, without /v1:
ANTHROPIC_BASE_URL=http://localhost:8765 ANTHROPIC_API_KEY=<PROJECT_KEY> claude
from anthropic import Anthropic
client = Anthropic(base_url="http://localhost:8765", api_key="<PROJECT_KEY>")
msg = client.messages.create(model="default", max_tokens=256,
messages=[{"role": "user", "content": "How many vacation days do I get?"}])
print(msg.content[0].text)
Endpoints, roles per endpoint and the error envelope are in the API reference.
How It Works
client ──► POST /v1/chat/completions (Bearer <PROJECT_KEY>)
│
├─ project → model connection (provider credentials stay server-side)
├─ project → RAG store (optional): embed query → top-k chunks → inject context
└─ adapter: OpenAI · Anthropic · Gemini · DeepSeek · Ollama · custom OpenAI (vLLM…)
JSON and SSE streaming, normalised to the OpenAI schema
│
└─ PostgreSQL + pgvector: users, connections (AES-256-GCM keys), projects,
documents (bytea), chunks, HNSW vector indexes, request logs
The binary is pure Go and ships as a static executable. Schema changes are embedded SQL files applied at startup under an advisory lock, so several replicas can start against one database.
Configuration
Configuration is environment variables. The ones most installations touch:
| Variable | Default | Purpose |
|---|---|---|
DATABASE_URL |
required for -app image |
PostgreSQL connection string; the all-in-one image uses its embedded server when unset |
SECRET_KEY |
DATA_DIR/secret.key fallback |
64 hex characters; encrypts provider credentials at rest |
PORT |
8765 |
HTTP listen port |
ADMIN_USER, ADMIN_PASSWORD |
admin, none |
Create the first administrator on an empty database |
TRUST_PROXY_HEADERS |
false |
Take the client address from X-Forwarded-For; enable only behind a reverse proxy |
TRUSTED_PROXY_CIDRS |
none | Networks whose proxy headers are honoured |
SECURE_COOKIES |
false |
Mark the session cookie Secure unconditionally |
ALLOW_PRIVATE_UPSTREAMS |
false |
Let provider URLs point at private networks |
PRIVATE_UPSTREAM_ALLOWLIST |
none | Hostnames allowed to resolve to private addresses |
Every variable, default and the -healthcheck / -version flags are in
Configuration.
Deployment
- State is the
/datavolume plusSECRET_KEY. In the all-in-one image/data/pgis the PostgreSQL cluster and/data/ragmuxholds thesecret.keyfallback. With the split layout the gateway is stateless and the state is thepgdatavolume plusSECRET_KEY. - Terminate TLS in front. Put a reverse proxy before the gateway and set
TRUST_PROXY_HEADERSandSECURE_COOKIES; see Behind a reverse proxy. - Back up. Use
scripts/backup.sh(pg_dump -Fc, both layouts) or the scheduledbackupCompose profile; see Backup and restore. - Scale out with the
-appimage and an external PostgreSQL; see Scaling.
Upgrading
Back up first, then pull and restart:
scripts/backup.sh && docker compose pull && docker compose up -d
With docker run, scripts/backup.sh does not apply; dump the database directly, then pull,
recreate the container on the same volume with the same SECRET_KEY (a new key cannot
decrypt stored credentials). See Backup and restore.
docker exec -u postgres ragmux pg_dump -Fc --no-owner --no-privileges -d ragmux > ragmux.dump
Migrations run at gateway start. The PostgreSQL major version inside the all-in-one image
never changes within a minor series, and pulling a newer image never runs pg_upgrade; see
Deployment layouts. Release notes are in
CHANGELOG.md.
Project Status
Beta. Ragmux is pre-1.0: the public API, configuration keys and metric names can still change between minor versions, and each change is listed in the CHANGELOG. The direction after the current release is in ROADMAP.md; OCR for scanned PDFs, SSO / OIDC login, cost-denominated budgets and an import tool for 0.1 databases are not scheduled.
Contributing
Bug reports, documentation fixes and pull requests are welcome; see CONTRIBUTING.md for the development setup and the pull request process.
License
Licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later). If you run a modified version as a network service, the AGPL requires you to offer its users the corresponding source.
The fonts in web/fonts/ are third-party and licensed under the SIL Open Font License 1.1;
the licence texts sit next to them.