Overview
The mcp-local-models server lives at mcp-local-models/server.py in the Grid repo. It runs as a Python process on ArcHelm, exposes five MCP tools over HTTP at port 8020, and proxies each tool call through ArcRouter's model aliases to the correct on-prem inference node.
The server uses streamable-http transport so it works as a standard MCP remote server.
Routing rule
The routing decision tree for any drafting task in this codebase:
| Task type | Use this tool | Model alias | Served by |
|---|---|---|---|
| Mechanical code / JSON / config | ask_coder | coding | qwen2.5-coder:32b (ArcRig3) |
| Analysis, gap analysis, synthesis | ask_analyst | analysis | qwen3-fast (ArcRig3) |
| UI / design / long markdown drafts | ask_designer | design | qwen3-fast (ArcRig3) |
| Quick classification / short answers | ask_chat | communications | llama3.1:8b (ArcRig1/2) |
Skip local routing when: the task needs knowledge of the current branch, session state, or tool results from the active conversation; the output needs project-specific entity names, voice, or context that can't fit in the prompt; or expected output is heavy prose (150+ lines). Route those directly in the main Sonnet session.
For Agent tool calls, default to model: "haiku" (API-billed traffic). Reserve Sonnet for the main Max plan session only.
Tools
All four ask_* tools accept a prompt string and an optional max_tokens integer override. The response is the raw model completion as a string.
Configuration
The server reads two environment variables:
| Variable | Default | Purpose |
|---|---|---|
| LITELLM_BASE | http://172.17.0.1:4001 | ArcRouter base URL (Docker host gateway default) |
| MCP_AUTH_TOKEN | (empty) | Bearer token for inbound requests — leave empty to skip auth |
The 172.17.0.1 default resolves to the Docker host gateway, which works when the MCP server runs inside a container that needs to reach ArcRouter on the host. For bare-metal installs, point this at ArcHelm's IP and port 4001.
Claude Code setup
Add to .mcp.json in the repo root or to the global Claude Code MCP config:
After adding the server, the five tools become available in Claude Code as mcp__local-models__ask_coder, mcp__local-models__ask_analyst, etc.
The server must be running on ArcHelm. To start it:
Timeouts and fallback
When a local model call times out or returns an error, note the failure and fall back to the main Sonnet session rather than retrying. Do not silently swallow the error. The recommended pattern:
Timeouts are expected during ArcRig maintenance windows. A cold model load on ArcRig3 adds 15-30s to the first request after idle.