Skip to content
shiva.wm
--:--

fleet-stats

real-time fleet analytics across two databases, fused

productiondata / analytics2026 — present

An operations dashboard that fuses data from two unrelated datastores — a document store of human-review events and a relational store of a distributed crawler's job, grid and output state — into one Fastify service whose metrics are computed live on every request. Because no single database owns both sides of the relationship, cross-cutting metrics such as review coverage, unreviewed backlog and per-query review bias are derived by reconciling identifiers across the two stores in the application layer.

Each endpoint fans out its workload concurrently — seventeen parallel SQL queries for crawler and dataset statistics, eight document-store aggregation pipelines for reviewer activity, and a mixed pass that builds a per-worker search-coverage matrix — then reshapes the results into chart-ready payloads, deriving behavioural worker metrics (tenure, active-day cadence, repeat ratio, average daily hours) and back-filling sparse aggregations into dense 60- and 90-day series and hourly heatmaps.

The frontend ships as a single dependency-free template rendering thirty-one visualizations — twenty-nine Chart.js charts plus two custom CSS-grid hourly heatmaps. Written in strict, modular TypeScript with pooled database access and graceful dual-connection lifecycle handling.

stack
TypeScriptNode.jsFastifyMongoDBPostgreSQLChart.jsCSS-grid heatmapsdate-fns