Fine-tuning, with receipts

Teach a small local model your docs.
Then prove it learned.

A knowledge pack is a versioned, portable artifact — curated sources, schema-validated training JSONL, a contamination-guarded held-out eval set, and a signed-shaped manifest. The before/after eval harness ships inside the format, because loss going down is not knowledge going in.

packsmith build packs/amber-framework      # chunk → split → generate → guard → emit
packsmith verify packs/amber-framework     # hashes, schemas, contamination — fail closed
packsmith train packs/amber-framework --base mlx-community/SmolLM-135M-Instruct-4bit
packsmith eval  … --out base.eval.json     # identical harness, before and after
packsmith diff  base.eval.json adapter.eval.json --fail-below-delta 0.10

Measured, not vibed

One M1 Max, one afternoon, per-item reports checked in. These are lab notes, not benchmarks — the eval questions are synthetic and not yet hand-reviewed, and our own spec forbids headlining synthetic-only numbers. With that stated first: in the measured run, recall of trained facts (normalized-substring match at temperature 0, under held-out question templates) went 1/20 → 11/20, while the held-out-source control set — whose units contributed zero training examples — showed no measurable gain (9/50 → 7/50). Training took 62.77 s at 0.42 GB peak, fully local.

The same experiments also caught two failure modes no loss curve shows: the default learning rate that walks validation loss from 4.94 to 1.96 while recall of trained facts barely moves (2/20 vs base 1/20 — same data, same iterations as the run above), and a cloze-only dataset that injected facts while collapsing multiple-choice format compliance to 1/30. Both are written up with full numbers and caveats — the eval questions are synthetic and not yet hand-reviewed, so read everything here as lab notes and run the harness yourself.

Read the guide

Built in Crystal, on Llamero

packsmith compiles on stock Crystal and runs its whole spec suite against Llamero's deterministic MockBridge — no models, no keys, green in CI. Generated records parse into real Crystal classes and enums — typed validation at parse time, not a best-effort post-hoc filter. Why that matters.

Does grammar-constrained decoding compose with a knowledge-pack adapter?

One measured data point (July 2026, llamero branch gbnf-structured-generation, SmolLM-135M, N=25/cell, matched prompts, calibrated field-subset oracle — methodology, deviations, and the raw text of every attempt: benchmarks/gbnf/BENCH_RESULTS.md). The adapter is a rank-8 llamero-docs knowledge pack that was not trained on these extraction tasks:

The two mechanisms fixed different failures in this run: the adapter had taught the model to stop (its unconstrained flat speed already matches grammar's), but couldn't make records structurally complete; the grammar guarantees the required keys exist but doesn't teach brevity. Stacked, grammar kept its 25/25 oracle pass rate on the tuned model — though not for free everywhere (on the deepest task, adapter+grammar ran ~16% slower than grammar-on-base, 975 vs 842 ms). One adapter, one model family, one machine: a composition data point, not a law. This measurement is llamero's, not packsmith's — the toolchain itself still makes no grammar-constrained-decoding claims.