Fetching posts
// GLAD LABS · DEV DIARY
The diary.
Daily founder notes from building Glad Labs — raw progress, decisions, and what broke today.
Dev Diary entries
The Timeout That Fired in Zero Milliseconds
The tell was buried in a GlitchTip stack trace: Timeout passed=90.0, time taken=0.001 seconds. A timeout that fires in a millisecond isn't…
The Scorer Was Silently Dropping Half Its Shot Scores
Two alerts fired on the same post -- herorenderfallback on shots 4 and 8, visionscorerunavailable on 5 of 12 shots -- and pulling that…
The Probe That Deployed Dead
Today's most humbling shipment was a fix for a fix. brain/clockskewprobe.py landed in (PR #2212) to catch something real:…
The Zombie That Wasn't RUNNING
Content generation went dark for roughly two hours before anyone noticed -- 02:31 to 04:57, per the audit log -- and the worker looked…
Self-healing, silent failures, and the "dead, not inert" bug
The firefighter we shipped in PR #2124 ran dead, not inert on prod. We'd built a deterministic self-healing loop--detect → act → verify →…
SVG crashes, Loki case-sensitivity, and the fight for an authoritative prompt catalog
We spent a good chunk of today chasing "ghost" errors in the console, starting with the Sparkline emitted invalid SVG path on empty data…
Retiring Gen-1 TopicDiscovery and Hunting Invisible Stalls
We finally killed the Gen-1 TopicDiscovery orchestrator (PR #2061). It felt good to delete nearly 900 lines of legacy code and over 1,500…
Solving the GPU Pinning Saga and Gemma's Meta-Commentary
We spent today fighting a ghost in our GPU orchestration, starting with fix(llm): stop setting litellm.apibase global (PR #2082). We had…
Taming the cadvisor leak and cleaning up LLM garbage
We spent most of today fighting an OOM cascade that nearly took down the WSL2 VM, starting with our attempt to cap cadvisor memory so it…
PII leaks and the three-stage shakedown of X distribution
A routine audit surfaced our own email in the public mirror, landing a post on X took three shakedown fixes, and an audio red herring ended…
Closing the Feedback Loop and Fixing Silent Failures
We spent today closing the gap between human intuition and machine execution. For too long, when we rejected a draft via regenatgate…
Hunting Ghost 503s and Pipeline Halts
Our biggest fight today was with a series of silent failures that only appeared in the wild. We spent most of the day chasing "ghost"…
Shrinking the Footprint and Cleaning the Pipes
We finally stopped the flashing terminal windows on our desktop by wrapping DeployCheckoutSync in a VBS helper (run-hidden.vbs) to force…
Deterministic Citations and CI Gates for Atom Drift
We spent a good chunk of today fighting "hallucinated" authority in our content. In fix(citations): deterministically strip ungroundable…
Surgical Regens and the WSL2 Wedge
We finally shipped previewgate -- component-scoped regen (PR #1851), solving a friction point that had been eating our time for weeks.…
Fighting VRAM collisions and API drift
The "GPU metrics STALE" alarm in PR #1796 was a wake-up call--we weren't just missing data, we were blind. After our deploy-clone cutover,…
Fixing the GPU lock and taming the internal RAG sweep
The 2026-06-19 pipeline validation exposed a few ghosts in our machine, and today was about exorcising them. The biggest fight was with our…
Killing false alarms and fixing "lying" MP3 headers
We spent a good chunk of today fighting ghosts in our monitoring and audio. The most annoying was the recurring Prefect queue backlog false…
Shipped 30 PRs and 28 notable commits — 2026-06-18
The narrative writer was unavailable this run, so here's the plain changelog. We shipped 30 PRs and 28 notable commits today. Auto-compiled…
Fixing Media Narration with Per-Media Scripts and CTA
We tackled a significant issue in our media pipeline today by implementing per-media narration scripts and calls-to-action (CTA). This…
What we shipped on 2026-06-14
We closed out a milestone today by shipping the final operator-console phase, Phase 13 -- Revenue, mobile, docs (final), which completes…
What we shipped on 2026-06-13
We wrestled with re-labeling our QA panel to reflect the real rails, and it was a long time coming. This change has been in the making for…
Lock down page view inflation with origin checks
We shipped version 0.81 today, but the most satisfying work was patching a blind spot in our analytics worker rather than just rolling…
What we shipped on 2026-06-11
We shipped v0.76 today, but feat(alerting): externalized all alert thresholds to app_settings was our primary focus (PR #1374). The goal…
What we shipped on 2026-06-12
We wrestled with a root cause that had podcast episodes stuck in R2, invisible to the feed, for weeks -- it turned out the RSS feed was…
