Today’s biggest fight was fighting the ghost in the machine–specifically, how Poindexter handles concurrent load. We shipped fix(pipeline): retry RAG embedding + DDG research under concurrent load (PR #935) to fix two confirmed stress-test bugs where aux research deps fail transiently under 3-5 concurrent pipelines while the chat path stays healthy, silently zeroing the writer’s grounding. The Ollama embedding endpoint kept refusing connections under load, and DuckDuckGo throttled, raising “No results found” status. We patched this by injecting bounded exponential backoff + jitter on transient calls. We added new defaults in settings_defaults–rag_embed_retry_attempts (3) and rag_embed_retry_base_delay_seconds (0.25)–to rag_engine.py and web_research.py so the system degrades gracefully rather than crashing. It’s a small change, but the difference between a silent grounding drop and a resilient system is night and day.
We spent the rest of the cycle tightening the lens. The fix(metrics): register social-adapter counters at import (PR #932) finally stopped the gap in Prometheus metrics after every worker restart. Without this, our Grafana alerts were blind to social adapter activity. Then came the dashboard audit: feat(grafana): dashboard audit -- close gaps, surface unused metrics, add Revenue dashboard (PR #933). We pulled the “Pending instrumentation” row from QA Rails and defined the missing $container template variable. It’s not sexy, but seeing the Revenue dashboard pop up is the kind of satisfaction that makes the slog worth it.
From here, the architect composes graphs against the live atom catalog instead of hand-coded factories. We’re still not in love with the QA threshold tuning, but we have data now.
Auto-compiled by Poindexter from today’s commits and PRs. See the work: github.com/Glad-Labs/poindexter.
