Open the configuration of most document processing pipelines and you will find one number. A confidence threshold of 0.80 or 0.90, applied to every field on every document type. The developer picked it during UAT because it made the demo look clean. Nobody from the business was asked. Yet that number decides which invoices are paid without a human looking, which claims go to a reviewer, and how much of the promised straight-through processing actually happens. It is a business decision wearing a model setting's clothes.
What the number actually controls.
A confidence score is the engine's own estimate that it read a field correctly. The threshold is the line below which the field goes to a person. Move the line up and more fields go to review, fewer wrong values reach the system of record, and straight-through processing falls. Move it down and the reverse happens. No setting reduces review effort and error rate at the same time. The threshold is a trade, and the person who should make it is the one who pays for both sides.
That person is the process owner. They know what a wrong bank account number on a vendor master costs, and what a reviewer's hour costs. The developer knows neither. Thresholds belong in the solution design document, per field, with the process owner's name against them.
Why one threshold is always wrong.
Fields carry different risk. On a supplier invoice, a wrong invoice date is an annoyance. A wrong total is a mis-posting that reconciliation will probably catch. A wrong IFSC code or IBAN is money sent to the wrong account. A single threshold treats all three the same, so it is too loose for the account number or too strict for the date. The result is a queue full of fields nobody needed to check, next to a handful that should never have gone straight through.
| Field | Cost of a wrong value | Sensible starting posture |
|---|---|---|
| Invoice date | Low. Ageing reports slightly off. | Loose threshold, rely on validation rules. |
| Invoice total | Medium. Caught at reconciliation, costs rework. | Medium threshold plus arithmetic check. |
| Bank account number | High. Payment to the wrong party. | Strict threshold plus master-data match. |
| Free-text description | Low. Rarely drives a decision. | Do not threshold. Capture as read. |
Setting the thresholds with the process owner.
The workshop takes an afternoon and should happen before build, with a sample of real documents on the table. Work through the fields one at a time and answer four questions for each.
- What happens downstream if this value is wrong? Name the consequence in business terms: a mis-payment, a regulatory breach, a customer complaint, a rework ticket. Rank the field high, medium or low on that answer alone.
- Does a validation rule catch a wrong value anyway? Totals can be checked arithmetically. Identifiers can be matched against a master. A field with a strong downstream check can carry a looser threshold, because the rule and the score are two independent lines of defence.
- What does a review cost? A field a reviewer confirms in two seconds can afford a stricter threshold than one that needs a phone call to verify.
- What straight-through rate has the business been promised? The combined effect of all the thresholds has to deliver it. Add them up on the sample and see whether they do.
Write the outcome into the design document as a table: field, threshold, validation rule, review action, owner. Treat it as a controlled artefact. A threshold change should go through the same change process as a change to the bot's posting logic, because it has the same effect on what reaches the ledger.
Tuning from review data.
The human-in-the-loop queue is the most valuable dataset the programme owns, and most teams throw it away. Every time a reviewer confirms or corrects a field, record the field, the engine's value, its confidence, and the final value. After a few weeks you can answer the questions that matter.
- Where were reviewers agreeing with the engine? If they confirmed the invoice date unchanged in nearly every case above 0.70, a threshold of 0.90 was costing hours and catching nothing. Lower it.
- Where was the engine confidently wrong? A field with high scores and a high correction rate is the dangerous one. The score is unreliable for that field on that document type. Raise the threshold, add a validation rule, or both.
- Which document types or suppliers drive the corrections? Corrections cluster. One supplier's template, one scanner, one branch's photocopies. Fix those at source and the queue shrinks without touching a threshold.
Review this monthly for the first quarter after go-live, then quarterly. Bring the process owner the table they signed in the workshop, with two new columns: correction rate above the threshold, and review volume below it. The decision to move a number stays theirs. The evidence comes from operations.
Two mistakes to avoid.
Treating the score as a probability. A confidence of 0.90 does not mean a nine in ten chance the value is right. Scores are engine-specific, often poorly calibrated, and they shift when the model is retrained or upgraded. After any model change, re-run the review analysis before trusting the old thresholds.
Thresholding the document instead of the field. Some pipelines send the whole document to review if any field falls below the line. That turns one uncertain date into a full manual re-key. Route the field, show the reviewer the crop of the page where it was read, and let the rest of the document continue.
A threshold is a decision about which mistakes the business is willing to pay for. Make it with the people who pay.
If your pipeline runs on a single number nobody remembers choosing, the fix is a workshop and a report. We are happy to look at your field list and review-queue data and suggest where the thresholds should move.