Model OffloaderGenAI-driven query routing that stops paying cloud prices for easy questions.
A Generative AI router that sits beside your copilot or chat client. It caches repeats, answers easy questions with a local generative model, and only spends cloud AI tokens on the queries that truly need a frontier model.
Type a question.
Watch it decide where to send it.
This runs the product's real Generative AI routing heuristic, ported line for line from our production engine — not a staged animation. It scores the text, explains why, and shows which model would answer it.
0.10
One router.
Three honest outcomes.
No black box. The classifier is a transparent heuristic you can read in one file, and the safety net — escalation on a weak local answer — keeps a wrong guess from becoming a wrong final answer.
Exact and semantic
Repeat and near-duplicate questions are answered from SQLite with no model call at all. Semantic matching uses Ollama embeddings when installed, with a strict trigram fallback otherwise.
Transparent scoring
Length, hard cues ("design a", "refactor", "debug", "step by step"), easy cues ("what is", "translate", "define"), code blocks and multi-part asks move the score. Every verdict comes with reasons.
Cloud as the safety net
Hard queries go straight to the cloud model. An empty, refusal-shaped, or failed local answer escalates automatically — so "easy" is a starting guess, not a guarantee.
Your copilot.
Two new tools, nothing else changes.
As an MCP server it gives Copilot or Claude Code two tools: a free local/cached answer for easy sub-questions, and a stats readout. It never calls the cloud itself — that stays your copilot's job.
# Ask your copilot, once the MCP server is registered: > Use offloader_ask to define idempotent in one sentence. <- "An idempotent operation is one that can be repeated any number of times without changing the result..." [offloader: route=local, saved≈$0.0007] # Ask again — served from cache, 0 ms: > Use offloader_ask to define idempotent in one sentence. <- route=cache-exact # Hard questions are declined so your copilot answers them itself: > Use offloader_ask: design a distributed rate limiter... <- {"declined": true, "reason": "too complex for local"}
One click, once licensed.
Percent and time shown as it runs.
Once access is granted, the installer checks Python and Ollama, installs whatever is missing, downloads the models, registers the MCP server in every client it finds, and runs a real smoke-test question — with a live progress bar the whole way.
# after access is provided: Setup.bat # installs Python/Ollama if missing, pulls the models, registers the # MCP server in Copilot / Claude Code / Claude Desktop / Cursor, # shows percent + elapsed/ETA, then runs a real test question.
# one-click Setup.bat is Windows-only today; the CLI runs anywhere:
python -m offloader doctor
python -m offloader install --target claude-code
This is proprietary software, licensed on request — not open source, no public source access.
Part of the GCC & FDE
Generative AI product suite.
Contact us to license Model Offloader for your team or platform.