Designs by Duhart ← All infographics

Designs by Duhart · Solomon · six projects

From a paper to a pod

A transformer implemented from the Annotated Transformer in pure PyTorch — no HuggingFace anywhere — then carried the whole way: corpus, tokenizer, training, nine gates with the authority to block a release, promotion, container, Kubernetes, autoscaling, and a Go retrieval tier alongside.

The Solomon training-to-serving pipeline WikiText-103 is cleaned and deduped into 130 million training tokens with a data card recording contamination checks, tokenized with a SentencePiece BPE vocabulary of 8192, and used to train a 6.84 million parameter decoder-only transformer to validation perplexity 40. Nine evaluation gates must all pass before the checkpoint is promoted to serving.pt, an image is rebuilt against it, and it is rolled out to k3s where an HPA scales it to three replicas. STEP 1 · CORPUS WikiText-103 raw CC BY-SA · cleaned, deduped 130M train / 275K val / 314K test data card + contamination checks STEP 2 · TOKENIZER SentencePiece BPE vocab 8192, byte fallback 4.02 chars/token 0 UNK tokens STEP 3 · MODEL SolomonLM · 6.84M params d_model 256 · 6 layers · 8 heads tied weights · pre-norm KV cache proven = full recompute STEP 4 · TRAIN Noam + label smoothing 8,000 steps · 3.0 h · CPU only MLflow · SHA256 manifest val PPL 40.0 STEP 5 · THE GATE — ALL NINE, OR NOTHING SHIPS 9 evaluation gates PPL 90.0 · BPC 1.62 · token-acc 24.9% · distinct-2 0.95 · repetition 0.5% LAMBADA 1.3% · HellaSwag 24.7% — both loaded from original sources, not a hub serving p95 454 ms · 116 tok/s — latency and throughput are gates, not footnotes all green → promote STEP 6 · PROMOTION artifacts/checkpoints/serving.pt model card + HTML report written rollback = an image tag, not a retrain STEP 7 · IMAGE container rebuilt against the promoted checkpoint lineage: corpus → run → image STEP 8 · SERVE k3s + Helm · HPA → 3 replicas FastAPI · SSE + WebSocket · continuous batching Prometheus · RS256 JWT · NetworkPolicy Alongside — Solomon retrieval tier · 4,757 lines of Go chunk · embed · search · store · bounded agent loop · grounding guard (with its own test) · Kafka ingestion · deployed as phase2-retrieval

The image running in the cluster is provably the checkpoint that passed, traceable through a SHA256 manifest back to the training run and the corpus it used.

The nine gates

All nine must pass. Not most. The two in the accent colour are the ones most candidates leave out of an evaluation story.

90.0test perplexityis it worse than the last one?
1.62bits per charactertokenizer-independent sanity
24.9 %token accuracynext-token prediction works
0.95distinct-2output has lexical variety
0.5 %repetition ratecatches degenerate loops
1.3 %LAMBADAlong-range context is used
24.7 %HellaSwagcommonsense continuation
454 msserving p95fast enough to ship
116 tok/sthroughputcheap enough to ship
Why latency belongs in a quality gate

A model that is accurate and too slow is not shippable, and discovering that after rollout is the expensive way to learn it. Folding serving p95 and throughput into the promotion gate means "good enough" is defined once, on both axes, before anything reaches a cluster. It is also why continuous batching was a design requirement rather than an optimization — static batching makes the first request in a window wait for the window to close, and that shows up directly in gate 8.

The six Solomon projects

Project 19

The Transformer

Every component verbatim from the reference or with its deviation justified in writing. Three deviations, three defenses — and one of them is "this is not actually a deviation."

Project 20

Corpus & Tokenizer

130M tokens cleaned and deduped, licence recorded, contamination checked against the eval suites. BPE with byte fallback: 0 UNK, and that zero is measured.

Project 21

Training & Tracking

Noam, label smoothing, checkpoint/resume, MLflow, per-run SHA256 manifests. CPU-only, and the constraint is documented rather than implied away.

Project 22

Evaluation Gates

Nine gates that can block a release, two of them performance. Benchmarks loaded from original sources so the contamination check means something.

Project 23

Serving & Kubernetes

SSE and WebSocket streaming over one engine, continuous batching, KV cache, HPA to 3 replicas, dry-run-validated manifests, a NetworkPolicy.

Project 24

Retrieval & Agent Tier

4,757 lines of Go: index, tools, a bounded loop, and a grounding guard with its own test — the answer to "what stops your RAG system making things up?"

The defects, published unprompted

CODE_REVIEW.md and BLOCKERS.md list what is still wrong, ranked, with nobody asking. This is the file to bring to an interview.

  1. SSE per-token decode drops inter-word spaces — on the server and the WebSocket path.
  2. The engine swallows generation exceptions → an empty 200 followed by [DONE]. Worse than a 500, because every client reads it as success.
  3. Disconnect cancellation is dead code → generations leak when a client hangs up.
  4. make deploy under sudo strips KUBECONFIG.
  5. --resume restarts the shuffle — a resumed run is not identical to an uninterrupted one.
  6. Prompt-trim edge case at max_tokens ≥ max_len − 1; a web-client double onDone; a localStorage seed-clobber race; flaky eval-smoke gates; sampler and SSE-parser duplication.
And what is blocked rather than done

Cloudflare Tunnel hostnames are dashboard-managed, so the public hostname → the in-cluster service and the API hostname → its in-cluster service are recorded in k8s/INGRESS.md as manual steps this host cannot perform. Phase 2 must restart its general-member to load the promoted weights. iOS builds need a macOS Xcode. Separating "finished" from "blocked on something outside this box" — with the exact hostname and port — is the same habit as the gap registers everywhere else in this portfolio.