If you have used a document extraction system, you have probably met a confidence score: a number that says how sure the system is about what it read. A 0.95 on an invoice number says the system is nearly certain; a 0.62 on a description says it may have misread.
CargoLint started there and moved on. Every document now carries one Quality score, built from the problems CargoLint finds on it, and every field carries a colour that follows the same rule. This post explains how the score is calculated, what the colours mean, and what sends a document to a person.
Two Numbers That Are Easy to Confuse
There are two different things behind the scenes, and they answer different questions.
- How clearly the AI read a value. When the AI extracts a document, it rates each value it reads: 0.95 when it is clearly legible, 0.75 when it may be garbled, 0.50 when it is guessing. It rates each line, each party’s name, address and tax ID, and the key header fields such as the invoice number, date and total.
- What is wrong with the document. The Quality score. It counts the problems CargoLint found: a value the AI may have misread, a field the document lacks, totals that do not add up, an HS code that does not fit the goods.
A reading rating is one input. A low one becomes a problem of its own, an unclear read, and from there it is treated like any other problem.
What Counts as a Problem
Four sources raise problems:
- The reading - how the document was read: an unclear read, a value the AI returned that is not in the document’s text, an identifier read with low certainty, line arithmetic that suggests a misread.
- The document checks - what the document says or lacks: a missing field, totals that do not add up, an Incoterm that does not exist, the tax a route implies, a sanctions match, a CITES species, a destination’s required invoice elements.
- The HS verdicts - whether each declared code is recognised and fits the description.
- CargoLint itself - notes about where a document came from, such as a packing list generated from an invoice.
Every problem type has one level, kept in one list, and that level is the same wherever the problem appears:
- Needs fixing - the document cannot do its job, or a regulator would stop it: no shipper or consignee, an invoice with no total or no lines, a transport document with no number, an HS code that is missing, wrong or not recognised, a value the AI is guessing.
- Warning - something a person should look at: a missing invoice number or address, totals that do not add up, a partial sanctions match, an HS code that fits only loosely, a value that may be garbled.
- Note - context with no cost: who carries the duty and tax under the Incoterm, a domestic shipment, a missing Incoterm, an HS code checked at six digits.
How the Quality Score Is Calculated
Every document starts at 100%. Each problem takes off its cost, and the costs compound:
- a warning costs 10%
- a problem that needs fixing costs 20%
- a note costs nothing
One warning gives 90%. Two warnings give 81%. One problem that needs fixing gives 80%. One that needs fixing and a warning give 72%.
HS code problems count once per document, at their worst level. An invoice with fifteen lines and no HS codes is one problem that needs fixing, not fifteen, so the score still says something about the rest of the document.
Hover over the score and it shows what it is made of: each problem and what it cost.
One Rule for Every Colour
The colours mean the same thing on the document, on every field, on every line and on every HS code:
- Green - nothing wrong
- Amber - a warning
- Red - something needs fixing
A field takes the colour of the worst problem that names it. An amber document has at least one warning and nothing that needs fixing; an amber field is where to look first.
What Sends a Document to Review
A person signs off every document before it is complete. What changes is whether it waits in Pending Approval for a quick confirm or sits in Review for a closer look. A document goes to Review for:
- Problems that add up - one problem that needs fixing, or two warnings, whichever source raised them: the reading, an unclear read, the document checks or the HS codes
- HS codes to confirm - codes are never applied automatically
- A degraded scan
- A blocking finding - an exact sanctions match, a CITES Appendix I species, or a check that could not run
- A shipment finding - a conflict between this document and another in the same shipment
With the costs above, one problem that needs fixing or two warnings is the same as a Quality score below 90%. So a document waiting in Pending Approval scores 100% or 90% and is never red: it has at most one warning to glance at. A problem in the paper itself counts like any other. An invoice that really has no invoice number cannot be fixed by reading it again, but a person still looks at it and decides what happens next, for example asking whoever issued it for a corrected copy.
Why Not a Confidence Threshold?
A single threshold looks simple, but it mixes two questions. A document can be read perfectly and still lack its invoice number, and a document can be complete and still have a smudged quantity. One number with one cut-off cannot say which, and a threshold tuned for one kind of document lands differently on another.
The Quality score keeps the questions apart. The problems say what is wrong, the colours say how much it matters, and a document goes to Review when its problems add up to one that needs fixing, or two warnings. The 90% the score shows is a consequence of that rule, not a cut-off tuned by hand.
Calibration Still Matters
A reading rating is only useful if it means something. A well-calibrated rating of 0.75 should mark values that turn out wrong more often than values rated 0.95. Whether the AI’s ratings behave that way is something reviewer corrections will show over time.
Unclear reads count like any other problem today: a value the AI is guessing, or two values it says may be garbled, send a document to Review. If the corrections show those ratings predict errors more or less often than expected, their level can be adjusted in the one list every part of CargoLint reads.
Learning From Corrections
Every correction made in review is logged with the document and field it applied to.
- HS code corrections build your organization’s classification history. When a document declares a code your team has confirmed more than once for a near-identical description, CargoLint stops flagging that code as doubtful. The history confirms declared codes; it does not rank or choose suggested ones.
- Extraction corrections accumulate as training data used to evaluate and improve extraction over time, unless you turn that off in your profile settings.
Common Misreadings
Reading the score as accuracy. An 81% document is not 81% correct. It has two warnings. Every value on it may be exactly what the paper says, and the warnings may be about the paper.
Treating amber as wrong. Amber means look here. Many warnings are confirmed in seconds: the address really is missing, the Incoterm really is unusual.
Expecting a field to show two levels. Each problem type has one owner and one level, so the same missing field is never a warning in one place and a problem that needs fixing in another.
A Worked Example
Three invoices arrive in the same shipment.
- Invoice A reads cleanly, but its line totals do not add up to the invoice total (a warning) and one declared HS code fits its description only loosely (a warning). It scores 81% and shows amber. It goes to Review: two warnings are enough, and its HS code waits for confirmation too.
- Invoice B reads cleanly except one line the AI rated as possibly garbled (a warning). It scores 90%, the line shows amber, and it waits in Pending Approval for a quick look and a one-click approval.
- Invoice C has an invoice number the AI returned but could not find in the document’s text (needs fixing). It scores 80%, shows red, and goes to Review, because a reader should check that number against the page.
Each document says what is wrong with it, in the same colours, and the reason it waits where it does.
CargoLint gives every document a Quality score built from the problems it finds, in one set of colours from the document down to each field, so a clean document is a quick confirm and the time goes to the documents that need it.