clawql-inference

Status: Shipped (July 2026) Package: packages/clawql-inference

clawql-inference is ClawQL’s TypeScript-native inference gateway and model-improvement platform — a LiteLLM-class layer with ClawQL’s trust model: WORM-auditable routing, semantic cache, model tier escalation, agent coordination hooks, verdict-filtered export, and fine-tuning flywheel. It is designed as a drop-in OpenAI replacement (OPENAI_BASE_URL=http://127.0.0.1:8080/v1) while closing the production loop that generic proxies leave open.

Related docs: Inference provider plugins · Token efficiency (12 layers) · Ouroboros library

What It Does

Quick Start

Example policy + walkthrough: docs/examples/inference/ (policy.yaml + README).

export CLAWQL_HOME="${CLAWQL_HOME:-$HOME/.clawql}"
mkdir -p "$CLAWQL_HOME/Inference"
cp docs/examples/inference/policy.yaml "$CLAWQL_HOME/Inference/policy.yaml"

export OPENAI_API_KEY=sk-...
export OLLAMA_BASE_URL=http://127.0.0.1:11434

clawql inference policy show          # expect policy_source: manifest+env
clawql inference serve --port 8080

Point clients at OPENAI_BASE_URL=http://127.0.0.1:8080/v1. Manifest YAML merges at runtime (resolveInferenceEffectiveEnv); env vars override YAML on conflicts.

One-shot CLI (no HTTP):

clawql inference complete --model ollama/phi4 --message "hello"

Architecture Overview

Entry Points

Gateway Decorator Stack (inner → outer)

Each complete() call passes outward through these layers before reaching a provider:

1. ConfiguredInferenceGateway — resolves provider/model, invokes the provider adapter
2. FallbackChainGateway — tries alternate models on primary failure (when fallback enabled)
3. SemanticCachedGateway — embedding-similarity cache lookup (when semantic cache enabled)
4. TokenEfficiencyGateway — Layers 3–11 (terse, compress, route, prompt-cache, structured/prefill markers)
5. EntitlementEnforcedGateway — plan limit checks via clawql-payments (when enforcement enabled)
6. ObservedInferenceGateway — appends an InferenceRecord to the call store
7. TracedInferenceGateway — OTLP spans to infra + Langfuse when configured

Call order from outside in: Traced → Observed → Entitlement → Efficiency → Cache → Fallback → Configured → provider.

Data Flow Summary

HTTP clients → auth middleware → OpenAI router → [Observed → Entitlement → Cache → Fallback → Configured] → provider plugin
CLI / library callers ──────────────────────────► [same decorator stack] ────────────────────────────────► provider plugin
Ouroboros loop ─────────────────────────────────► ConfiguredInferenceGateway (innermost) ───────────────► provider plugin

ObservedInferenceGateway ──writes──► Inference store (jsonl / postgres / memory)

Gateway Decorator Stack

createInferenceGateway() composes decorators in this order (inner → outer):

const gateway = createInferenceGateway({ env: process.env })
const result = await gateway.complete({
  model: 'anthropic/claude-sonnet-4',
  messages: [{ role: 'user', content: 'Summarize this spec' }],
  correlationId: 'seed_abc_gen_2',
  team: 'eng',
})

Disable layers via options: \{ semanticCache: false \}, \{ fallback: false \}, \{ store: null \}.

HTTP API Layer

Entry Points

OpenAI-Compatible Endpoints

Gateway ladder (8.0.0)

On the Managed Edge Gateway the same host exposes four rungs — /v1 → /mcp → /memory → /decision — with shared virtual keys and WORM identity. Spec: gateway-ladder-v0.1.md.

Erasure destroys the working-tree note, purges memory.db / pgvector / ontology.db, and crypto-shreds the per-note encryption key so git history and R2 mirrors retain ciphertext only. WORM logs opaque pathId + content hash (never readable path or body). Erased content hashes land on .clawql/erasure-deny.json; export jobs skip them. Details: gateway ladder § Erasure · memory-obsidian § Erasure.

Opt-in chat enrichment: virtual-key memoryEnrichment outranks x-clawql-memory-enrich / CLAWQL_INFERENCE_MEMORY_ENRICH (default off).

Request Headers

Drop-In OpenAI Client

export OPENAI_BASE_URL=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=<virtual-key-or-upstream-key>
npx clawql-inference

Provider Plugins

Built-ins register via composeDefaultProviderPlugins() unless allow/denylisted:

Model ids use provider/model (e.g. ollama/phi4, anthropic/claude-sonnet-4). Bare public ids like gpt-4o resolve through the registry when a matching provider is configured.

Third-party plugins use the same InferenceProviderPlugin contract. Subpath export: clawql-inference/plugin.

Model Tier Escalation

TierEscalationRouter implements AdaptiveRouter with three tiers.

Escalation is off by default unless CLAWQL_INFERENCE_ROUTING_ENABLED=1 or CLAWQL_INFERENCE_MODEL_PIN is set. CLAWQL_INFERENCE_MODEL_PIN=<modelId> bypasses the ladder.

Tier map overrides persist at $CLAWQL_HOME/Inference/tier-map.json (written by clawql inference finetune register or escalation set-tier).

clawql inference escalation show
clawql inference escalation set-tier --tier frugal --model ollama/phi4-custom

Ouroboros Integration

When EvolutionaryLoop runs with an AdaptiveRouter: router.initialTier() picks starting tier per seed / decomposed-child context. On generation failure (AC fail, low eval score, drift exceeded), router.escalate() moves one notch up. model_escalation audit events append to the Ouroboros Postgres / in-memory event store.

Semantic Cache

Layer 5 in token efficiency. Enabled with CLAWQL_INFERENCE_SEMANTIC_CACHE=1.

Cache hits return stored responses with cacheHit: true on inference records. Embedding failures fail open to live inference.

clawql inference cache   # show active config

Fallback Chains

When the primary provider/model fails, configured alternates are tried before surfacing an error.

export CLAWQL_INFERENCE_FALLBACK_ENABLED=1
export CLAWQL_INFERENCE_FALLBACK_FRUGAL=ollama/phi4,openai/gpt-4o-mini
export CLAWQL_INFERENCE_FALLBACK_STANDARD=groq/llama-3.3-70b,anthropic/claude-haiku-4

Chains also persist at $CLAWQL_HOME/Inference/fallback-chains.json. Responses include fallback.attempted and fallback.succeeded.

clawql inference fallback

Virtual Keys and HTTP Auth

Per-team API keys with optional USD budgets and rate limits.

export CLAWQL_INFERENCE_KEYS_ENABLED=1
clawql inference keys create --team eng --budget-usd 500 --rate-limit 100rpm
clawql inference keys list
clawql inference keys revoke --id vk_abc123

Keys persist at $CLAWQL_HOME/Inference/virtual-keys.json (secrets stored as SHA-256 hashes only). When enforcement is active, /v1/* requires a valid key; /healthz stays open.

team and virtualKeyId are stamped on inference records; spend rollups support --group-by team.

Plan Entitlements and Payments

The inference HTTP server integrates with clawql-payments for four independent billing modes:

Do not conflate plan usage (usage.json), inference call-store tokens (clawql inference spend), and virtual-key USD budgets — see Three usage systems.

export CLAWQL_PAYMENTS_ENFORCE_INFERENCE=1
export CLAWQL_PAYMENTS_REPORT_STRIPE_METER=1
export STRIPE_METER_EVENT_NAME=clawql_inference_calls
export CLAWQL_X402_ENFORCE=1

clawql payments plan show
clawql inference serve --port 8080

Inference Call Store

Every successful completion writes an InferenceRecord used by observability and export.

Backends

Key Record Fields

Observability CLI

clawql inference logs [--model M] [--tier T] [--since 24h] [--limit 50]
clawql inference spend [--group-by model|tier|team|provider] [--since 7d]
clawql inference trace --correlation-id <id>

Export and Fine-Tuning Flywheel

Production traffic
  → WORM-logged inference (prompt, response, tier, verdict, correlation_id)
  → Evaluator verdicts + quality filters (+ OKF v0.2 verified/status filters)
  → PII-scrubbed dataset export (JSONL or PorTAL bundle)
  → Fine-tuning job (Anthropic / OpenAI) **or** PorTAL task-latent + alignment
  → Custom model / LoRA registered in ModelTierMap
  → Deployed to Frugal tier
  → Better cheap-tier results → better verdicts → better training data

PorTAL (Ramp Labs): ClawQL intends to make Flywheel adapters portable across base models via task-latent + per-base alignment. See PorTAL + Intelligence Flywheel for staged CLI shapes (--format portal-bundle, finetune refit) and trade-offs.

Local / Argo GPU training (SFT → DPO → GRPO → SPIN): When RTP/OBT traces from Harvey LAB (or OpenBench) land in the training bucket, the method-aware pipeline formats datasets, schedules Unsloth/TRL jobs, and promotes domain adapters. Spec + scaffold: Training Pipeline v0.1. GRPO is the strongest fit for Harvey LAB because rubric criteria are verifiable rewards.

Export

clawql inference export \
  --output ./training-data/export.jsonl \
  --verdict passed \
  --tier frugal \
  --format openai-jsonl \
  --min-score 0.8

PII scrubbing (Presidio) is on by default. Every export writes a WORM dataset manifest (sample hashes, filter criteria, Merkle root, policy version).

Erasure deny-list: when a vault note is erased, its content hash is appended to .clawql/erasure-deny.json. Export loads that deny-list (from CLAWQL_OBSIDIAN_VAULT_PATH / --vault) and drops records whose message or response body hashes match, so erased content never reappears in a future training set. Historical export files already on disk remain out of band — lineage shows which past exports and fine-tuned models included the content so those can be regenerated if required. See gateway ladder § Erasure.

Fine-Tune and Register

clawql inference finetune \
  --dataset ./training-data/export.jsonl \
  --base-model gpt-4o-mini \
  --provider openai

clawql inference finetune status --job-id ftjob_abc123
clawql inference finetune register --job-id ftjob_abc123 --tier frugal --alias ollama/phi4-production-v3

Pipeline Automation

Scheduled auto-export when sample thresholds are met.

clawql inference pipeline enable \
  --schedule "0 2 * * 0" \
  --min-samples 500 \
  --verdict passed \
  --target-tier frugal \
  --base-model gpt-4o-mini

clawql inference pipeline status
clawql inference pipeline run          # manual trigger
clawql inference pipeline worker       # cron sidecar

Config persists at $CLAWQL_HOME/Inference/pipeline.json. Cron worker: CLAWQL_INFERENCE_PIPELINE_WORKER=1 starts with inference serve.

Agent Coordination

TierEscalationRouter.shouldTriggerAgentCoordination() fires when combined drift exceeds 0.3, or when standard-tier is exhausted with active failure signals.

When CLAWQL_INFERENCE_AGENT_COORDINATION_ENABLED=1: evaluateAgentCoordination() builds an agent_coordination audit entry. Hermes stub runs when HERMES_BASE_URL is unset; live MoA when configured. Ouroboros appends the event to the lineage store alongside model_escalation.

Policy

clawql inference policy show [--json]

resolveInferencePolicy() aggregates the effective view from manifest YAML ($CLAWQL_HOME/Inference/policy.yaml or CLAWQL_INFERENCE_POLICY_MANIFEST) merged with environment variables — env wins on conflicts. Source is manifest+env when a manifest is loaded, otherwise env.

Example manifest (full copy: docs/examples/inference/policy.yaml):

policyVersion: '2026.07.01'
inference:
  escalation:
    enabled: true
    tierMap:
      frugal: ollama/phi4
  cache:
    enabled: true
    threshold: 0.92
  pipelineWorker:
    enabled: true
    pollMs: 60000
  observability:
    profile: external

Multi-instance serve workers use Postgres advisory locks (pg_try_advisory_lock) keyed by pipeline schedule + UTC minute when CLAWQL_INFERENCE_DATABASE_URL is set, so only one replica runs each cron tick.

Persistence Layout

All operator state under $CLAWQL_HOME/Inference/:

CLI Wiring

clawql inference <subcommand>
├── serve              HTTP gateway (+ optional pipeline worker)
├── complete           One-shot completion
├── logs | trace | spend
├── export | finetune [status|register]
├── escalation [show|set-tier]
├── pipeline [enable|status|disable|run|worker]
├── cache | fallback | policy
└── keys [create|list|revoke]

Environment Variables

Full env table: clawql-inference.md.

Package Exports

Differentiation vs LiteLLM

References

• Inference provider plugins
• Token efficiency architecture
• Ouroboros library

© Copyright 2026. All rights reserved. · ClawQL on GitHub