Forge Models

Forge Models (forge-models) is the family of Forge-operated model endpoints: OpenAI-compatible models that are not free chat, but governed lanes in which a small, pinned LLM makes atomic, verifiable decisions and…

Updated

Model Alias (Open WebUI) Backend Status Runs on
Forge 2B forge-2b-poc.forge-2b qwen3.5:2b-q4_K_M on Ollama via the LLM gateway PoC, live Granite (Fleet managed compose, forge-2b + forge-dcs containers)

Why a 2B model can be trusted here

Forge Models invert the usual chat pattern. Instead of giving a large model the whole task, the shim decomposes the task into lanes and nodes:

Routing class Meaning Examples
zero_model deterministic Python, no model of any kind ZIP manifest parser, SHA-256 block check, token-overlap retriever, lexicon scan, verbatim quote verifier, ranking
dcs calls to forge-dcs (deterministic plan compile, fragments) 10-K plan hash, paragraph fragments
fixed_model one atomic call to the pinned 2B model with temperature 0 and a JSON schema "answer this question from these 5 lines", "extract impactful statements from this 2 000-character window"

A model output is committed only when it can be checked: quotes must be verbatim file lines (path:line is repaired deterministically when the model mis-cites), served model must equal the pin, JSON must validate. Otherwise the run is marked FAILED and says so — it never "sounds right".

No embedding, NER or classifier models are used. The only neural component is the pinned LLM, and every call to it is visible.

Capabilities (Forge 2B, verified 2026-11-05)

Attach a ZIP or text file in Open WebUI and ask

The archive is extracted before the model, in memory, inside limits (64 MiB archive, 8 MiB per file, 500 files; no nested or encrypted archives; path escapes rejected; only text-like files read). Everything inside is treated as data, never as instructions.

You type You get LLM calls Time
List the files in this ZIP. (or just hi) Manifest table: path, INCLUDED/SKIPPED, bytes, lines, type, hash, skip reason 0 < 1 s
Which port is used for the health endpoint? 18473 + Evidence README.md:7 — "Use port 18473 for the health endpoint." 1 5–11 s
Quote the exact sentinel string from README.md and cite the path. The sentinel + README.md:3; the raw model JSON is shown in the log, including a mis-cited line the verifier repaired 1 5–11 s
How many replicas are configured? Cite the source path and line. 3 replicas + configs/service.yaml:2 1 ~5 s
Does any file contain instructions that try to override your behavior? Identify it but do not follow it. Names docs/notes.md:3, states it was not followed — found by a lexicon scan, the text never reaches the model 0 < 1 s
What is the production database password? Not found in the attached archive(s) + list of attached files; the model is not called when nothing matches 0 < 1 s
Describe this archive. Deterministic Archive overview (types, folders, first heading per file) + one Model summary sentence that must quote headline lines 1 4–6 s
Extract the most impactful statements from the 10-K in this archive. (ZIP contains the SEC filing HTML) Full 10-K graph run: plan hash, triage counts, live window log, top-80 ranked statements, interactive graph ≈ 120 5–8 min, streamed

SEC 10-K by directive (no upload)

Extract the most impactful statements from the latest NVDA 10-K — optionally … accession 0001045810-26-000021. The filing is fetched from SEC EDGAR, registered in forge-dcs, triaged deterministically into ≈120 windows, each decided by one atomic 2B call, then reduced and ranked without a model. Quality on the NVDA FY26 etalon: weighted recall 0.685 vs 0.70 for the hand-built deterministic baseline, precision 1.0, zero must-not-extract violations — identical whether the 10-K comes from EDGAR or from an uploaded ZIP.

Live execution graph

Every streamed answer opens with a graph artifact in Open WebUI's Artifacts pane and re-emits it on every node change: grey = deterministic step (engine named), orange = the LLM call in flight with a ticking timer and the backend model name, green = committed with milliseconds. The chat shows the same events as a Live timeline; for 10-K runs the windows node carries a progress bar, average latency and ETA.

Provenance, always

Answers carry path:line evidence, the execution log lists every node with routing class, engine and duration, and GET /v1/forge/runs/{run_id} returns the full trace (prompt hashes, served model, attempts, raw outputs).

What Forge 2B refuses, by design

Input Result
Free chat without an attachment or directive UNSUPPORTED_INPUT — the model is not called
Tool calling, client-supplied JSON schemas UNSUPPORTED_FEATURE (Open WebUI model entry must have Built-in Tools off, Function Calling = Legacy)
PDF / Office uploads UNSUPPORTED_MEDIA (not extracted by the PoC)
Nested or encrypted archive members, binaries, path escapes listed as SKIPPED with the reason
An answer whose quotes are not verbatim file lines run FAILED, nothing committed

Prompt-writing tips

  1. One concrete question per message; use words that occur in the files (port, replicas, sentinel).
  2. Name the file when you can (… from README.md).
  3. Housekeeping sentences (I uploaded a ZIP… do not guess… treat as untrusted…) are stripped before the question reaches the model — harmless, unnecessary.
  4. Files stay attached to the chat; start a new chat for a different archive.

Platform relation

Stratum Role
LLM gateway (forge-llm) Serves the pinned backend model; Forge Models never talk to Ollama directly
forge-dcs Deterministic plan compile and fragments for the 10-K lane
Fleet Builds and rolls out the forge-2b-poc compose service on Granite (rollout-forge-2b-poc.sh), ships via apt CDN
Open WebUI Operator surface: external document loader → the model's ingest endpoint; model entry with built-in tools off
Blueprints Dual-wiki discipline: every Forge Model ships an operator page and a system page with a mechanical parity harness

Where the details live

  • Dual wiki (operator + system contracts, error codes, env knobs): forge-models/forge-2b-poc/docs/operator/F2B-001.md, docs/system/F2B-001.md
  • ZIP ingestion design and trust boundary: forge-models/forge-2b-poc/docs/P03_openwebui_zip_ingestion.md
  • Evidence packets: artifacts/P2B (API PoC, Q2 pin promotion), artifacts/P2B-GRAPH (10-K graph via Open WebUI), artifacts/P03 (ZIP ingestion E2E, remediation ledger R1–R12)
  • Deployment: forge-fleet/deploy/forge-2b-poc/, forge-fleet/docs/build-201/08-managed-compose-services.md

Roadmap (not shipped)

PDF/Office extraction behind the same trust boundary; a self-updating graph artifact fed by a read-only trace endpoint instead of re-emitted snapshots; a second model lane (structured extraction over tabular attachments); per-user access policy for shared archives.