Skip to content

Model Routing and Execution Architecture

Why this chapter matters

Model routing is the discipline of choosing an execution component without confusing the component for the intelligence, authority, or mission it serves. This chapter helps humans understand substitution and helps AI readers preserve scope across runtime changes.

Continue to Memory and Knowledge Storage Architecture to follow what the routed work leaves behind.

Defines bounded, attributable, replaceable model selection and execution without treating a model as an authority.

  • INFRA3-R001: Routing SHALL record model identity, version, purpose, inputs, policy, resource scope, and accountable owner.
  • INFRA3-R002: Execution SHALL isolate tools, secrets, memory, and external effects from untrusted model output.
  • INFRA3-R003: Routing SHALL fail closed or require review when model identity, provenance, policy, or safety state is unknown.
  • INFRA3-R004: Model capability, confidence, or output SHALL not create authority or permission.
  • INFRA3-R005: A route SHALL state data sensitivity, tool boundary, evaluation basis, fallback limit, and exit condition before execution is proposed.
  • INFRA3-R006: Model substitution, fallback, or retraining SHALL preserve purpose, scope, provenance, policy, and review evidence.
  • INFRA3-R007: Routing review SHALL test prompt injection, data leakage, correlated error, unsafe tool use, stale metadata, and silent fallback.
  • INFRA3-R008: A routing design SHALL NOT deploy a model, expose credentials, or create external effects; it remains conceptual and reviewable.

This Draft excludes live model deployment and provider selection.

Routing decision record

Each routing decision should compare capability, provenance, data sensitivity, latency, cost, uncertainty, tool access, failure behaviour, and exit options. The record shall identify why a route was selected and what observation would trigger re-routing or human review.

Failure cases

Unverified model substitution, prompt or data leakage, tool escalation, stale model metadata, correlated model error, and silent fallback are material failures. Fallback shall preserve the original limits and shall not silently increase authority or data access.

Design evidence

Routing review should record the candidate set, sensitivity class, isolation boundary, provenance state, tool permissions, fallback limits, evaluation evidence, and exit condition. A model response is untrusted output even when the route is technically healthy.

Operating model and evidence

Routing begins with purpose, consequence, data class, and required capability. It compares candidate models by provenance, version, isolation, tool access, evaluation evidence, failure behaviour, latency, cost, and exit path. The route records why a candidate was selected and which conditions require review, containment, or a different route.

Fallback preserves the original limits and does not silently increase context, permissions, or external reach. Reviewers test untrusted output handling, prompt and data isolation, correlated model error, metadata freshness, and recovery from an unavailable provider. Model output remains evidence or advice until an authorised process evaluates it.

Interpretation cases

  • Conforming: Route purpose, sensitivity, model identity, isolation, evaluation, fallback, and exit are recorded.
  • Prohibited: Capability or confidence becomes authority.
  • Boundary: An unavailable model causes bounded fallback or safe pause.
  • Failure: Unknown provenance or tool state blocks execution and preserves the record.
  • Loophole: Fallback silently increases data access or authority.
  • Misuse: Untrusted output is used to expose secrets or contact external systems.
  • Care-control: Routing supports a person proportionately while preserving privacy, consent, and review.