Document Accuracy Scoring: What the Numbers Mean for Your Immigration Case

When AI analyzes your documents, it often provides a confidence score or accuracy score—usually a percentage like "87% confidence this is a valid passport" or "92% match with required format." These numbers feel objective and scientific, but understanding what they actually mean is crucial. A 92% score might mean different things depending on what's being scored.

There are several types of scores, and conflating them creates confusion:

Extraction accuracy measures how confident the AI is that it correctly read information from your document. If your passport is perfectly clear and digital, extraction might be 99% accurate. If it's a photocopy of a handwritten document, it might be 78% accurate. This score tells you: "How much do I trust this data has been correctly pulled from the document?" Anything below 85% should probably be manually verified.

Format compliance scores measure whether your document matches expected standards—does it have all required sections, is information in the right fields, does it follow the official template? This is more binary; you're either compliant or not. If the score is 95%, it likely means the document is missing or misaligned on a few minor elements.

Consistency scores measure how well information in this document matches other documents in your file. If every document lists your name differently, consistency might be 68%. This is valuable because it highlights real problems you need to resolve.

Risk or flag scores measure how likely a particular element is to cause problems with your application. An AI might score your employment letter as "medium risk (63%)" because the job title doesn't align with industry standards, but this is probabilistic—it's saying "in 63% of similar cases, this type of discrepancy caused delays," not "this document is 63% wrong."

Here's the critical insight: All these scores are relative to their training data. An AI trained primarily on North American documents might give a lower compliance score to a document formatted according to European standards—not because it's actually wrong, but because it's unusual in the training data. This is a limitation, not an error, but it matters for how you interpret the score.

Numbers create false precision. A score of 87% sounds more scientific than "probably okay, but have a human look at it." In reality, both statements might be saying the same thing. Scores below 75% usually warrant careful review. Scores between 75-90% suggest you should check the AI's work but probably don't indicate serious problems. Scores above 90% suggest high confidence, but human review is still wise before submission.

One practical note: High scores from extraction don't guarantee high scores for compliance. A document might be perfectly read (high extraction accuracy) but not meet the specific format required by the immigration office (lower format compliance). These are independent measures.

The human element remains essential. An AI might give your employment letter a 94% compliance score because it has all the right elements, but an immigration officer might question the letter because the salary listed seems low for the role. The AI scored format compliance, not reasonableness—that's a human judgment call.

Try this: Next time an AI gives you a confidence score on a document element, ask yourself: What exactly is being scored? Is it extraction accuracy (was the text read correctly), format compliance (does it meet requirements), or risk assessment (is this likely to cause problems)? Understanding the type of score changes how you should act on it.