Application isolation
Each credential belongs to one active organization, workspace, environment, and project. Keys are displayed once, stored only as hashes, immediately rotatable, and constrained by a public model alias, private provider mapping, rate, queue, concurrency, request, output, timeout, and retention limits.
Durable queue and useful progress
AES-256-GCM encrypted jobs receive a stable FIFO sequence, queue position, estimated wait, estimated run duration, and completion estimate. Bounded workers prevent shared-model overload; callers can poll, retrieve results, or cancel, while authorized tenant roles and the protected platform administrator receive content-free history views.
A deliberately narrow production boundary
The current slice supports non-streaming text chat completions through one operator-managed llama-server bound to host loopback. Tools, multimodal input, streaming, RAG, provider failover, semantic guardrails, budgets, and a cost ledger remain unshipped until they have independent security and backpressure designs.
Frequently asked questions
Can applications connect directly to llama.cpp?
No. The production provider stays on 127.0.0.1:18080; public traffic enters the existing gateway /ai/ path and reaches the policy/queue service through a Unix socket.
Does RLX Gate claim provider-neutral AI routing today?
No. The shipped capability is the governed shared-Llama slice described above. Provider registries, failover, streaming, RAG, and budget controls remain backlog.