Batch-fix plan — nightly rollup workbench/c/batch-fix-plan-workbench100%
markdown
Batch-fix plan — nightly rollup row drop
Root cause: the aggregator's row-batching step slices its in-memory array without a bound, so any customer clearing 10,000 events/day silently loses every row past the cut — no error, no log line.
Plan
- Find the unbounded slice in the aggregator's batching step
- Bound it at the real row count instead of an implicit array-length cap
- Add a regression test at 10,001 rows (one past the old silent cutoff)
- Open a PR against main; dispatch to the workstation box for the fix
Done means: sandbox gates green, regression test passes, 48-72h production soak shows zero drops across every over-10k account.
note
Nightly batch job silently drops rows over 10k
While triaging support ticket #4821 I found the nightly usage-rollup job truncates any customer with more than 10,000 events in a day — no error, no log line, just missing rows. Looks like an unbounded array getting sliced somewhere in the aggregator.
- no open questions yet — what must this design answer?