Your fraud rate looks low because the losses have not arrived yet.

A recent group of approvals has had less time to produce reported fraud than an older group. Compare cohorts at similar ages, keep unknown outcomes visible, and connect later losses to the decisions that allowed the activity.

Otherwise, the newest policy can appear strongest simply because nobody has finished reporting its losses.

Separate the decision clock from the outcome clock

Store when the decision happened, when the activity occurred, when a concern was first reported and when the outcome was confirmed. These dates answer different operational questions.

Stripe's dispute-measurement guide (opens in a new tab) distinguishes dispute-date activity from charge-date rates and explains that rates can change as later disputes arrive. The same measurement problem can arise wherever your own confirmed outcomes lag behind activity.

Do not borrow a card-dispute waiting period for every product. Estimate observation patterns from the outcomes relevant to your business.

Compare groups with equal time to mature

Build cohorts around a defined decision period and keep material differences visible: product, market, customer type and policy version.

For an illustrative comparison, assess two approval cohorts after the same number of days. Report how many outcomes are confirmed and how many remain unknown. A newer cohort can be useful early evidence, but it is not a completed performance verdict.

Track count-based and value-based loss measures separately. One large loss and many small losses can imply different operational problems.

Keep the policy that made the decision

Retain the original inputs, decision, rule or workflow version and relevant evidence references. Attribute a later confirmed outcome to that historical decision, even if today's customer status has changed.

Use stable outcome identifiers to avoid counting repeated reports as new losses. Record corrections and recoveries separately rather than deleting the history. Define whether a metric uses gross loss, recovered amounts or net loss.

Unknown, reported, confirmed and overturned outcomes should remain distinguishable. Do not turn every account without a fraud report into a training example labelled legitimate.

Make the next change from evidence

Review mature outcomes alongside approval rate, review workload and genuine-customer impact. Investigate where a policy helped and where it merely delayed a decision.

Toden's decision-quality guide explains how confirmed downstream outcomes support evaluation. Confirm that your integration sends the supported outcome data. Bring one well-defined cohort to a monitoring discussion before treating a green dashboard as proof that a rule is safe.