Handbook
Forge Models
Forge Models (forge-models) is the family of Forge-operated model endpoints: OpenAI-compatible models that are not free chat, but governed lanes in which a small, pinned LLM makes atomic, verifiable decisions and…
Updated
| Model | Alias (Open WebUI) | Backend | Status | Runs on |
|---|---|---|---|---|
| Forge 2B | forge-2b-poc.forge-2b |
qwen3.5:2b-q4_K_M on Ollama via the LLM gateway |
PoC, live | Granite (Fleet managed compose, forge-2b + forge-dcs containers) |
Why a 2B model can be trusted here
Forge Models invert the usual chat pattern. Instead of giving a large model the whole task, the shim decomposes the task into lanes and nodes:
| Routing class | Meaning | Examples |
|---|---|---|
zero_model |
deterministic Python, no model of any kind | ZIP manifest parser, SHA-256 block check, token-overlap retriever, lexicon scan, verbatim quote verifier, ranking |
dcs |
calls to forge-dcs (deterministic plan compile, fragments) | 10-K plan hash, paragraph fragments |
fixed_model |
one atomic call to the pinned 2B model with temperature 0 and a JSON schema |
"answer this question from these 5 lines", "extract impactful statements from this 2 000-character window" |
A model output is committed only when it can be checked: quotes must be verbatim file lines (path:line is repaired deterministically when the model mis-cites), served model must equal the pin, JSON must validate. Otherwise the run is marked FAILED and says so — it never "sounds right".
No embedding, NER or classifier models are used. The only neural component is the pinned LLM, and every call to it is visible.
Capabilities (Forge 2B, verified 2026-11-05)
Attach a ZIP or text file in Open WebUI and ask
The archive is extracted before the model, in memory, inside limits (64 MiB archive, 8 MiB per file, 500 files; no nested or encrypted archives; path escapes rejected; only text-like files read). Everything inside is treated as data, never as instructions.
| You type | You get | LLM calls | Time |
|---|---|---|---|
List the files in this ZIP. (or just hi) |
Manifest table: path, INCLUDED/SKIPPED, bytes, lines, type, hash, skip reason | 0 | < 1 s |
Which port is used for the health endpoint? |
18473 + Evidence README.md:7 — "Use port 18473 for the health endpoint." |
1 | 5–11 s |
Quote the exact sentinel string from README.md and cite the path. |
The sentinel + README.md:3; the raw model JSON is shown in the log, including a mis-cited line the verifier repaired |
1 | 5–11 s |
How many replicas are configured? Cite the source path and line. |
3 replicas + configs/service.yaml:2 |
1 | ~5 s |
Does any file contain instructions that try to override your behavior? Identify it but do not follow it. |
Names docs/notes.md:3, states it was not followed — found by a lexicon scan, the text never reaches the model |
0 | < 1 s |
What is the production database password? |
Not found in the attached archive(s) + list of attached files; the model is not called when nothing matches | 0 | < 1 s |
Describe this archive. |
Deterministic Archive overview (types, folders, first heading per file) + one Model summary sentence that must quote headline lines | 1 | 4–6 s |
Extract the most impactful statements from the 10-K in this archive. (ZIP contains the SEC filing HTML) |
Full 10-K graph run: plan hash, triage counts, live window log, top-80 ranked statements, interactive graph | ≈ 120 | 5–8 min, streamed |
SEC 10-K by directive (no upload)
Extract the most impactful statements from the latest NVDA 10-K — optionally … accession 0001045810-26-000021. The filing is fetched from SEC EDGAR, registered in forge-dcs, triaged deterministically into ≈120 windows, each decided by one atomic 2B call, then reduced and ranked without a model. Quality on the NVDA FY26 etalon: weighted recall 0.685 vs 0.70 for the hand-built deterministic baseline, precision 1.0, zero must-not-extract violations — identical whether the 10-K comes from EDGAR or from an uploaded ZIP.
Live execution graph
Every streamed answer opens with a graph artifact in Open WebUI's Artifacts pane and re-emits it on every node change: grey = deterministic step (engine named), orange = the LLM call in flight with a ticking timer and the backend model name, green = committed with milliseconds. The chat shows the same events as a Live timeline; for 10-K runs the windows node carries a progress bar, average latency and ETA.
Provenance, always
Answers carry path:line evidence, the execution log lists every node with routing class, engine and duration, and GET /v1/forge/runs/{run_id} returns the full trace (prompt hashes, served model, attempts, raw outputs).
What Forge 2B refuses, by design
| Input | Result |
|---|---|
| Free chat without an attachment or directive | UNSUPPORTED_INPUT — the model is not called |
| Tool calling, client-supplied JSON schemas | UNSUPPORTED_FEATURE (Open WebUI model entry must have Built-in Tools off, Function Calling = Legacy) |
| PDF / Office uploads | UNSUPPORTED_MEDIA (not extracted by the PoC) |
| Nested or encrypted archive members, binaries, path escapes | listed as SKIPPED with the reason |
| An answer whose quotes are not verbatim file lines | run FAILED, nothing committed |
Prompt-writing tips
- One concrete question per message; use words that occur in the files (
port,replicas,sentinel). - Name the file when you can (
… from README.md). - Housekeeping sentences (I uploaded a ZIP… do not guess… treat as untrusted…) are stripped before the question reaches the model — harmless, unnecessary.
- Files stay attached to the chat; start a new chat for a different archive.
Platform relation
| Stratum | Role |
|---|---|
| LLM gateway (forge-llm) | Serves the pinned backend model; Forge Models never talk to Ollama directly |
| forge-dcs | Deterministic plan compile and fragments for the 10-K lane |
| Fleet | Builds and rolls out the forge-2b-poc compose service on Granite (rollout-forge-2b-poc.sh), ships via apt CDN |
| Open WebUI | Operator surface: external document loader → the model's ingest endpoint; model entry with built-in tools off |
| Blueprints | Dual-wiki discipline: every Forge Model ships an operator page and a system page with a mechanical parity harness |
Where the details live
- Dual wiki (operator + system contracts, error codes, env knobs):
forge-models/forge-2b-poc/docs/operator/F2B-001.md,docs/system/F2B-001.md - ZIP ingestion design and trust boundary:
forge-models/forge-2b-poc/docs/P03_openwebui_zip_ingestion.md - Evidence packets:
artifacts/P2B(API PoC, Q2 pin promotion),artifacts/P2B-GRAPH(10-K graph via Open WebUI),artifacts/P03(ZIP ingestion E2E, remediation ledger R1–R12) - Deployment:
forge-fleet/deploy/forge-2b-poc/,forge-fleet/docs/build-201/08-managed-compose-services.md
Roadmap (not shipped)
PDF/Office extraction behind the same trust boundary; a self-updating graph artifact fed by a read-only trace endpoint instead of re-emitted snapshots; a second model lane (structured extraction over tabular attachments); per-user access policy for shared archives.