On this page6
Ask a vendor how good their fraud model is and you’ll often hear “99% accurate”. That figure sounds impressive and tells you almost nothing. If only 1 in 100 transactions is fraudulent, a model that approves everything is also 99% accurate, and catches no fraud at all.
Fraud is a lopsided problem
Real fraud is rare, which makes it hard to learn from. There are very few examples compared with the mass of normal activity, and fraudsters change their approach as soon as one gets blocked. Any system that treats it like an ordinary classification task will look good in testing and disappoint in production.
The numbers that matter
- Recall: of all the real fraud, how much did we catch?
- Precision: of everything we flagged, how much was genuinely fraud?
- False positive rate: how many good customers did we annoy?
- Cost per decision: what does a missed fraud cost versus a blocked legitimate order?
The right balance depends on your business. A high-value electronics store can accept more manual review than a cheap subscription service where every blocked sign-up is lost revenue.
Rules and models work best together
Rules are fast, easy to explain and good at known patterns, such as the same card used on ten accounts in an hour. Machine learning is better at subtle combinations of signals that no one wrote a rule for. We usually combine both, then add a middle tier: clear cases are approved or blocked automatically, and borderline ones go to a human reviewer.
Explain every flag
An analyst who sees “risk score 0.87” can’t do much. An analyst who sees “new device, shipping address differs from billing, three failed payments in ten minutes” can decide in seconds. Reason codes also help when a legitimate customer asks why they were blocked.
Plan for drift
Fraud patterns shift, so a model that was sharp in spring may be weak by autumn. We monitor the inputs and the score distribution, alert when behaviour changes, and keep a clear process for retraining with the labels your reviewers produce.
Speed matters
For checkout or login, scoring has to come back in milliseconds. That affects design decisions early, such as which data can be looked up in real time and which is only available in a nightly batch.
If fraud losses or false alarms are a problem, tell us what you see today and what data you collect. We’ll outline an approach and what it would cost.
You can read more about our Fraud & Anomaly Detection service on the main site.
