How to evaluate receipt verification software: a practical scorecard
Build a representative receipt test set and measure extraction, product matching, duplicate detection, review load, and decision quality.
A receipt-verification demo usually starts with a clean receipt and ends with a green tick. That proves the happy path exists. It does not tell you how the system will behave when the campaign receives a faded thermal receipt, a cropped total, an unfamiliar retailer abbreviation, or the same purchase photographed twice.
A useful evaluation starts with the decision you need to make. For a loyalty campaign, that decision may be: does this submission contain an eligible product, within the campaign window, and has the purchase already been used? Reading the text is only one part of the job.
Build a test set that resembles the campaign
Do not assemble a folder of whichever receipts happen to be nearby. Define the expected mix before selecting examples. Include the retailers, languages, currencies, image sources and product categories that the live campaign will encounter. Keep difficult but legitimate submissions: long receipts, folds, glare, faint printing, abbreviations and several similar variants of the promoted product.
Add known negative cases deliberately. A synthetic evaluation set might contain:
| Case | Ground truth | What it tests |
|---|---|---|
NORD MKT / SPK WTR 6X1.5 |
Eligible six-pack | Abbreviation and pack-size matching |
| Receipt photographed twice, once cropped | Same purchase | Visual and purchase duplicate checks |
| Correct product, purchase one day early | Ineligible | Date-window rule |
| Eligible brand, wrong flavour and size | Ineligible | Lookalike product handling |
| Bottom edge missing | Not assessable | Capture-quality and abstention behaviour |
Label every expected field and decision before running a vendor. If reviewers disagree, resolve the disagreement and record the final label. Otherwise the benchmark measures differences between reviewers as if they were system errors.
Keep a holdout set. Examples used to configure fields, aliases and rules should not also be used to report the final result.
Score the layers separately
One headline “accuracy” number conceals the failure you need to fix. Score at least five layers.
Capture quality. Did the system reject evidence that could not support a reliable decision? Also count readable receipts rejected unnecessarily. A quality gate should reduce avoidable review, not become an unexplained rejection machine.
Field extraction. Measure exact or appropriately normalized agreement for merchant, date, total, currency and line items. Report a denominator for each field. A receipt with a correct total and missing date is not simply “90% accurate”; those fields have different consequences.
Product matching. Compare the extracted line against the campaign catalogue. Record correct matches, wrong matches and “uncertain” outcomes. An honest uncertain result routed to review is safer than a confident match to the wrong SKU.
Duplicate detection. Use known duplicate groups if you want to calculate precision and recall. Include recompressed, cropped and rotated copies, plus two different photos of the same purchase. If the set has no known duplicate groups, report what the test actually establishes instead of inventing a recall rate.
Final decision. Measure false approvals, false rejections and not-assessable/review outcomes. The business impact differs: a false approval leaks budget; a false rejection harms a genuine participant; a review outcome consumes time but preserves the decision.
Include operations and integration
Verification software also runs a queue and delivers results. Measure median and tail decision time, the share routed to review, reviewer handling time, and whether reasons are specific enough to act on. Test API retries with the same client submission identifier. Confirm webhook signatures, duplicate event handling and what happens after a temporary delivery failure.
For Open Loyalty programs, keep responsibilities explicit. Steve verifies the evidence and delivers approved purchase data. Open Loyalty applies campaign and earning rules and updates the member account. Your acceptance test should cover both sides without assigning points logic to the verification layer.
A scorecard that supports a decision
Use a weighted scorecard tied to the campaign risk:
- Representative coverage: retailers, languages, formats and difficult captures
- Extraction: field-level results with denominators
- Product matching: correct, incorrect and uncertain outcomes
- Fraud controls: duplicate scope and evidence shown to reviewers
- Decision quality: false approvals, false rejections and review rate
- Operations: reasons, queue actions and audit history
- Delivery: idempotency, signed events, retries and downstream status
- Commercial terms: what is billed, including unreadable uploads and reviewed exceptions
Ask every vendor to return the individual results, not only a slide with an aggregate. You should be able to inspect the source evidence, extracted fields, rule outcome and reason for each error.
Steve’s receipt-verification workflow combines structured extraction, configured rules, duplicate checks and an exception queue. The capture-quality gate stops blurry, dark or badly framed images before analysis, and gated submissions are not billed.
If you are evaluating receipt verification, book a demo and bring a labelled sample that includes the awkward cases. We will return the fields, verdicts and review reasons so you can score the workflow against your own standard.
