Backend performance
The sync job that took half a day
AssetSonar's device sync was taking 12 to 15 hours. I rebuilt the pipeline around batched writes and removed the redundant lookups.
- Role
- Backend engineer brought in to diagnose and redesign the production path
- Constraint
- A 12–15 hour sync could not keep its intended hourly schedule
- Decision
- Batch the unit of work, eager-load related data, and keep the downstream contract stable
- Result
- Under 1 hourMeasured across a 15-day production window; the pattern was reused in later MDM integrations.
Sanitized artifact
Record-by-record work became bounded batch work.
Device synchronization processing model before and after
Before
One record at a time
- Read one inventory record
- Load its related data
- Process and persist it
- Repeat the same lookups
Repeat per record
After
One bounded batch at a time
- Collect a bounded batch
- Eager-load related data once
- Process the batch
- Persist and move to the next batch
Repeat per batch
Ownership and reflection
My part
I profiled the existing job, redesigned the batching and query paths, and shipped the change without altering the downstream data contract.
The team's part
The backend team carried the pattern into later MDM integrations. I owned the original performance redesign.
Hardest trade-off
The quickest rewrite would have changed the payload shape. Keeping the contract stable made the rollout safer and the pattern reusable.
What I'd do earlier
The fix started too late. I would put query counts and runtime budgets beside the job before its first large workload, not after it had become a half-day dependency.
01 · Constraint
What was breaking
New hardware would not show up for most of a working day. The pipeline had accumulated N+1 queries and unbatched writes on a table carrying years of device history.
02 · Decisions
How I approached it
I traced the expensive query paths, rebuilt the work around bounded batches, and removed repeated lookups from the hot path. The redesign had to reduce runtime without changing the data contract downstream integrations relied on.
03 · Consequence
What changed
The same job now finishes in under an hour. More importantly, the batching pattern was reused in the MDM integrations added afterward instead of remaining a one-off performance patch.