Do we have to replace LiteLLM to use AnchorGate?

No — and you shouldn’t. AnchorProxy is OpenAI-compatible on both sides, so it chains in front of LiteLLM or behind it with a base-URL change. LiteLLM keeps routing and cost optimization; AnchorGate adds enforcement and evidence. Replacing a good router with a deliberately minimal one would make your stack worse.

Which order should the chain run in?

We recommend apps → AnchorGate → LiteLLM → providers: policy and PII redaction then run before the request fans out, so every upstream sees only redacted, admitted traffic. The reverse also works when you want routing decisions first — every request still crosses the gate and lands on the ledger.

LiteLLM has guardrails hooks — why add a separate gate?

Hooks are useful, and if they cover your needs, use them. AnchorGate is for when the requirement is stronger than a hook: enforcement that fails closed rather than open, a compiled human-approved policy bundle rather than per-config callbacks, and decision records that are cryptographically signed rather than logged. That’s a difference in kind, not a missing feature.

AnchorGate + LiteLLM

Route with LiteLLM. Govern with AnchorGate.

LiteLLM is a very good router — provider breadth, load balancing, budgets, spend tracking. AnchorGate doesn’t compete with any of that, and won’t pretend to. It adds the layer a router isn’t built for: deterministic policy in the path, redaction before egress, and evidence an auditor can verify. Both speak OpenAI-compatible, so they chain.

Get started — it's free View on GitHub

How it composes

One chain, both jobs

Point your apps at AnchorGate and set LiteLLM as its upstream: policy and redaction run before the request fans out to providers, and every decision lands on the ledger. The reverse order works too — either way, each layer does the job it was built for.

# apps → AnchorGate (governs) → LiteLLM (routes) → providers

export OPENAI_BASE_URL=http://anchorproxy:8080/v1

# anchorgate.yaml — LiteLLM is just an upstream

upstream: http://litellm:4000/v1

The difference in kind

Two jobs, done by the layer built for each

These rows compare purposes, not feature checklists — their features ship weekly and ours do too, and a stale checklist helps nobody.

LiteLLM

AnchorGate

LiteLLM

Built to optimize: unified access to a very large provider catalog, load balancing, retries, budgets, and cost tracking.

AnchorGate

Built to govern: deterministic policy verdicts, PII redaction, and MCP gating on the same OpenAI-compatible chain — fail-closed when it can’t decide.

LiteLLM

Routing intelligence is the product — picking the right upstream at the right cost is the job.

AnchorGate

Routing is deliberately minimal — configurable fallback only. Your router keeps that job; we’d rather say so than half-build it.

LiteLLM

Logs and callbacks feed your observability stack — operational visibility for engineers.

AnchorGate

Decision events are signed (ECDSA P-256), appended to an immutable ledger, and export as a Trust Packet your auditor verifies offline.

LiteLLM

Caching for cost and latency across workloads.

AnchorGate

Exact-match cache only, keyed on a SHA-256 of the normalized request — a hit is provably the same answer, which is the property evidence needs.

Characterization accurate as of July 2026. If we’ve gotten something about LiteLLM wrong, tell us and we’ll fix it.

The honest split

Where each one earns its place

Use LiteLLM for

Provider breadth — one API across a very large model catalog
Load balancing, retries, and cost-aware routing
Budgets, virtual keys, and spend tracking per team
Caching tuned for cost and latency

Add AnchorGate for

Fail-closed policy enforcement before requests egress
PII redaction before any provider sees the payload
Deny-by-default MCP tool-call security
Signed, offline-verifiable decision evidence and compliance posture

FAQ

Composing with LiteLLM

Do we have to replace LiteLLM to use AnchorGate?: No — and you shouldn’t. AnchorProxy is OpenAI-compatible on both sides, so it chains in front of LiteLLM or behind it with a base-URL change. LiteLLM keeps routing and cost optimization; AnchorGate adds enforcement and evidence. Replacing a good router with a deliberately minimal one would make your stack worse.
Which order should the chain run in?: We recommend apps → AnchorGate → LiteLLM → providers: policy and PII redaction then run before the request fans out, so every upstream sees only redacted, admitted traffic. The reverse also works when you want routing decisions first — every request still crosses the gate and lands on the ledger.
LiteLLM has guardrails hooks — why add a separate gate?: Hooks are useful, and if they cover your needs, use them. AnchorGate is for when the requirement is stronger than a hook: enforcement that fails closed rather than open, a compiled human-approved policy bundle rather than per-config callbacks, and decision records that are cryptographically signed rather than logged. That’s a difference in kind, not a missing feature.

Keep your router. Add the governor.

The chain is one base-URL change: your apps point at the gate, the gate points at LiteLLM. Enforcement, redaction, and evidence — without touching your routing setup.

Deploy the stack

or email hello@anchorgate.ai