Cutting 2,800 weekly alerts to 80 on a platform doing millions of transactions a day

The platform was not untested. It was over-alerted. Three scanners between them produced thousands of findings a week, the great majority of which were not exploitable, and the engineering organisation had quietly stopped reading them.

• Alert fatigue had become alert blindness

  A sampled week produced 2,800 findings. Fewer than 60 were reproducible. Engineering's rational response was to deprioritise the queue, which meant the 60 waited alongside the 2,740.

• Deploy velocity outpaced any review

  Around 200 deploys a day across 600 services. No review process gated on a human reading a report could keep up, so the report stopped being a gate at all.

• Checkout logic was tested by traffic, not by tests

  Promotions, gift cards, refunds and split payments interact. Nothing in the pipeline attempted to abuse those interactions deliberately. The flaws surfaced as unexplained revenue leakage weeks later.

• Ownership was ambiguous at 600-service scale

  A finding without an owner is a finding that ages. Routing depended on someone recognising the service name, which failed reliably for anything outside the top twenty services.

The core change was refusing to report anything that had not been exploited. Everything else (routing, prioritisation, the checkout work) followed from a queue small enough that a human could read all of it.

1. 1

   Validation before reporting

   Every finding

   Every candidate finding is exploited in a sandboxed replay before it is raised. Anything that cannot be reproduced is dropped rather than downgraded, which is what took a 2,800-item weekly queue to a two-figure one.

2. 2

   Business logic testing on checkout

   Continuous

   Concurrent redemption of the same gift card, promotion stacking, price manipulation between cart and capture, refund replay and inventory oversell were each modelled as an abuse case and run continuously.

3. 3

   Ownership resolved from the service catalogue

   Automatic

   Findings inherit the owning team from the deployment metadata, so a vulnerability in an obscure service reaches its team without anyone recognising the name.

4. 4

   Contextual risk scoring over raw CVSS

   Per finding

   Internet exposure, authentication requirement, data sensitivity and whether the path reaches payment infrastructure adjust the score, so an authenticated medium on the payment service outranks an unauthenticated high on the careers site.

5. 5

   Deploy-triggered scoping

   ~6 min

   A deploy scopes the next run to the changed service and its callers rather than re-testing 600 services, which is what makes per-deploy testing viable at 200 deploys a day.

Measured over the first six months, against the previous six months of multi-scanner output.

99.2%

Reproduced by the team on reviewThis engagement, first six months, re-run from the captured request

97%

Fewer findings raised2,800 per week to roughly 80, with more real issues among them

$2M

Estimated loss preventedGift card race condition, modelled on observed abuse rates

24/7

Continuous coverageAcross 600+ services and 12 storefronts

6 min

Median per-deploy scanScoped to changed services and their callers

2 hrs

Median time to ownerDown from 9 days for services outside the top twenty

• A finding you cannot reproduce is not a finding

  Reporting unvalidated candidates does not transfer risk to the engineering team, it transfers triage cost. Dropping them outright is what made the remaining queue credible enough to act on.

• Logic flaws are where the money is

  Not one of the highest-value findings had a CVE. The gift card race condition was a correctly patched, fully up-to-date service behaving exactly as written.

• Scope the scan to the change

  Full re-tests cannot run 200 times a day. Scoping to the changed service and its callers is what makes per-deploy security testing arithmetically possible.

• Routing is a security control

  Time-to-owner dominated time-to-fix at this scale. Automating the routing improved remediation more than any change to detection did.