MIT licensed · self-hosted · no account

Call a role,
not a model.

Model names leak into hundreds of call sites, and every swap becomes a migration. Muxaris puts one registry in front of your providers: your code asks for reasoning or structured, and the registry decides what actually answers — including what answers when the first choice doesn't.

Install in 30 seconds See the routing model

Runs in production behind a study platform serving real students — not a weekend benchmark.

app/summarize.py 
# No provider SDK. No model string. No fallback plumbing.
from muxaris import chat

reply = chat("structured").complete(
    messages=["role": "user", "content": prompt],
    response_format="type": "json_object",
)

# Primary answered in 1.2s.
# Had it timed out, the role's fallback would have —
# and this line would not have changed.
primary glm-4.7-flash fallback minimax-m2.5 budget 45s

One registry in front of every provider you already pay for

  • Amazon Bedrock
  • OpenRouter
  • NVIDIA NIM
  • Google Gemini
  • OpenAI
  • Ollama
  • Sarvam
  • Azure OpenAI

01 — The routing model

A role is a job to be done. The registry decides who does it.

Every gateway lets you pick a model. Muxaris asks a different question: what is this call for? Bind that answer once, in one file, and every call site inherits the decision — including the fallback and the latency budget that belong to it.

Role What it's for Primary Fallback Budget
flagship Long-form generation where quality is the whole point kimi-k2.5 minimax-m2.5 background
reasoning Multi-step planning, tool selection, hard analysis gpt-oss-120b minimax-m2.5 background
structured Strict JSON that a parser downstream has to trust glm-4.7-flash minimax-m2.5 background
workhorse High-volume classify, extract, label — cheap and fast magistral-small gpt-oss-20b 45s
guardrail Runs first on every turn, so it can never be the slow one gpt-oss-20b gpt-oss-120b 45s

Swapping a provider is a one-line diff in this table. No call site changes, no redeploy of application logic, no grep for the old model string.

02 — What you get

The plumbing you'd write anyway, already written.

01

Fallback is a property of the role

Every role declares its backup up front. When the primary times out, errors, or returns an empty body, the fallback answers on the same call — your caller never learns anything went wrong.

02

Latency budgets that match the caller

A guardrail running on every user turn cannot share a timeout with a nightly batch job. Interactive roles fail fast on a bounded client; background roles keep the long one.

03

Clients are built once

Provider clients are cached by role signature instead of reconstructed per call. Session and connection setup stops showing up in your p99.

04

JSON repair instead of lost work

When a model closes an object with the wrong bracket, a cheap repair model reconstructs it and logs a window around the offending character. Only broken replies pay the round trip.

05

Every call priced and logged

Tokens, latency, provider, role and cost land in one table per call — rolled up daily and purged on a retention window, so the log never becomes the thing you have to clean up.

06

Text, vision and streaming on one path

Image input and token streaming route through the same registry as plain completions. One integration, not three parallel ones that drift apart.

03 — The diff

Same behaviour. One-tenth the surface area.

Provider-direct repeated per call site 
import boto3, json, time

client = boto3.client("bedrock-runtime",
                      region_name="ap-south-1")

try:
    out = client.converse(
        modelId="zai.glm-4.7-flash",
        messages=msgs,
    )
except (Timeout, ClientError):
    # ...and now the fallback, by hand
    out = client.converse(
        modelId="minimax.minimax-m2.5",
        messages=msgs,
    )

body = json.loads(out["output"][...])  # may raise
# cost logging: TODO
With Muxaris written once 
from muxaris import chat

body = chat("structured").json(messages=msgs)

# Fallback:      from the role
# Timeout:       from the role
# Client reuse:  automatic
# Broken JSON:   repaired, not raised
# Cost + tokens: already in your usage table

Changing provider later means editing the registry row, not this file — or the ninety-seven others that call the same role.

04 — Accounting

Know which role is spending your money.

A provider invoice tells you what you spent. It won't tell you that one guardrail is firing on every turn, or that a role has been quietly serving from its fallback for a week. Muxaris writes a row per call — role, model, route taken, tokens, latency, cost — then rolls it up daily and drops the raw rows on a retention window you set.

  • Per-call rows with the route actually taken
  • Daily rollups so dashboards stay cheap to query
  • Automatic purge — the log never outgrows the app
usage_log schema sample
Role Model In / Out ms Route
guardrail gpt-oss-20b 412 / 96 380 primary
reasoning gpt-oss-120b 3,104 / 812 4,910 primary
structured minimax-m2.5 1,880 / 640 7,240 fallback
flagship kimi-k2.5 9,220 / 3,410 18,650 primary

Illustrative rows showing the shape of the table — not a benchmark.

05 — Get started

Self-hosted, MIT, no control plane.

Muxaris is a library, not a service. There is nothing to sign up for and no proxy sitting between you and your provider — your keys stay in your environment and your traffic never leaves your infrastructure.

1

Install 

pip install muxaris
2

Point it at a provider 

export MUXARIS_PROVIDER=bedrock
export AWS_REGION=ap-south-1
3

Call a role 

from muxaris import chat

chat("reasoning").complete(messages=msgs)
Read the source Back to the routing model