How to Judge a Face Recognition Accuracy Claim

Vendor material tends to lead with one figure — some number of nines, measured somewhere, on somebody's faces. It is not usually dishonest. It is just not a claim you can act on, because it collapses two different errors that have opposite causes and very different costs.

The two errors, and who feels each one

Every identification system can be wrong in exactly two directions, and the distinction is the whole subject:

The asymmetry in that last column is the important part. A false reject is loud and self-correcting — someone complains and it gets fixed. A false accept is quiet and permanent: nobody reports having been paid for a day they did not work, so the error enters payroll and stays there.

Which means a system tuned to keep employees happy is tuned in the direction that damages your data, and it will look like it is working well while doing so. This is worth knowing before someone adjusts a threshold to stop the complaints.

They trade against each other, always

Matching works on a similarity score against a threshold. Raise the threshold and you demand a closer match: fewer wrong people admitted, more right people turned away. Lower it and the reverse. There is no setting at which both improve, which is why "accuracy" as a single number is really a statement about where somebody set a dial.

So the question to a vendor is not "how accurate is it" but "at what false accept rate is that false reject rate measured, and can we change the threshold ourselves". A vendor who cannot express performance as a pair is quoting one point on a curve without telling you which point.

Why a good-sounding rate is a daily nuisance

Per-event error rates feel small and accumulate fast. Take a hypothetical site of 400 staff, each recognised twice a day, over a 22-day month — illustrative numbers, chosen to make the arithmetic plain. That is 17,600 recognition events:

A 3% failure rate reads as 97% accurate, which sounds respectable and is operationally unusable. This is the same trap as character-level accuracy in document OCR, where a per-character figure hides a per-field failure rate — the arithmetic that makes a 99% claim on cheque fields far less impressive than it appears.

Published figures do not describe your workforce

Benchmark results are measured on curated datasets under controlled capture. Your entrance is not that. Accuracy varies with lighting, camera angle, distance, motion, and how much of a face is visible — and in Gulf conditions that means direct sun at one hour and glare off pale ground at another, plus headwear, hard hats, glasses, masks and beards that are ordinary rather than exceptional.

Demographic variation is also real and well documented across the field: error rates differ by skin tone, age and gender, and a system evaluated mainly on one population can perform measurably worse on another. For a workforce as mixed as a typical Omani site, a headline figure from elsewhere tells you very little, and the only number worth trusting is one measured on your own people at your own doorway.