Because document AI extraction is the easy part. What separates a demonstration from a production system is disambiguation, meaning knowing that three different spellings are the same entity and that two identical spellings are not. Also knowing when the model should refuse to answer rather than risk it.
Most projects also begin without the one artefact that makes everything else measurable, namely a set of cases with the correct answer known, defined before anyone chooses a model.
We can speak with specific numbers. We rebuilt DIGITARQ for the Directorate-General for Books, Archives and Libraries. One search now covers 20 archives, across more than 8 million descriptive records and more than 63 million digitised images, corresponding to over 100 kilometres of physical documentation. We trained named entity recognition and disambiguation models for Portuguese archival metadata, on the Portuguese state advanced computing and supercomputing network. The full project write-up is in our case studies.
Document AI extraction, classification and entity recognition: what is the difference?
They are distinct tasks with distinct difficulties, and they are frequently sold as though they were one. Confusing them is why a budget comes up short.
Why is disambiguation harder than document AI extraction?
Because it requires a decision the model cannot take on its own: what counts as the same entity.
In DIGITARQ, the previous system treated "Torre do Tombo", "ANTT" and "Arquivo Nacional" as three different text strings. A person baptised in a parish record held in one archive and named in a will held in another were two unconnected records. The value of an archive lives in the connections, and the data was stored as isolated records and searched as plain text.
The enterprise version of the same problem looks like this. A single company appears as a supplier under four legal names, three historical tax numbers and two abbreviations. One employee shows up with and without a middle name, with and without accents. Contract references arrive in different numbering formats depending on who typed them. None of these cases is solved by a better model. They are solved instead by a business decision about what constitutes identity, written down before the project starts.
What do you need before starting a document AI extraction project?
In batch, in most cases, and this is the architectural decision with the largest impact on cost. In DIGITARQ, recognition and disambiguation ran as batch processing over the entire migrated dataset. Entities were extracted and resolved once, across all the data, and they feed the platform. The models are not invoked in real time as users browse.
Why does document AI extraction fail on Portuguese?
Because the linguistic variation is greater than what generic models were calibrated for, and our own experience came from the extreme version of that problem. Historical Portuguese is irregular, heavily abbreviated and inconsistently spelled across centuries. Off-the-shelf document AI extraction is not enough for that material, so we adapted and evaluated the models specifically for this domain, comparing modern transformer architectures to select the best performer on archival language.
That is the extreme, but the same logic applies well before it. A generic model recognises common Portuguese names and fails on the less common, which is exactly where errors hurt. Domain vocabulary, whether legal, technical or sector-specific, is not represented either, because the model sees words rather than concepts from your business. Old documents inside a modern company also follow different conventions, since twenty years of contracts contains several writing regimes.
So the practical conclusion is not always to train your own model, which is expensive and frequently unnecessary. Instead, evaluate every candidate on your own material before choosing.
What should stay with you after the project?
Four things, and none requires technology. They determine whether the project is measurable.
The first is the one most often missing and the most costly to lack. Building a reference set is annotation work by someone who knows the material, and it turns the rest of the project into a measurement rather than an argument.
When does this fall within the AI Act?
It depends on what you do with the result, not on the technology used, and the distinction is useful because it avoids unnecessary compliance scope. Extracting fields from an invoice and classifying documents for filing is not, in itself, a high-risk case. Using the result to make decisions about people can be. The Annex III categories of Regulation (EU) 2024/1689 include areas such as employment, credit assessment and access to public services, and those obligations were deferred to 2 December 2027 by Regulation (EU) 2026/1744, which entered into force on 27 July 2026.
The Article 50 transparency obligations, on marking generated content, apply to those who generate content. A system that extracts and classifies is not typically generating, but if the same product also produces summaries or text, that part is. In practice: classify the use, not the technology, and do that analysis at the start. Our summary of what changed in the AI Act in August 2026 sets out which obligations moved and which did not.
What should stay with you?
Five things, and the first is the one most often lost. The reference set and the evaluation results, which is the asset that lets you change supplier or model without starting over. The data model, including entity definitions and disambiguation rules, in a readable format. The extracted data, in a schema that does not depend on the platform that extracted it. The record of what was processed, when, and with which model version, without which you cannot know what needs reprocessing. And documentation of the decisions, including boundary cases.
A note on open standards. In DIGITARQ the graph was modelled on CIDOC-CRM, the ISO 21127 standard for cultural heritage information interoperability, with a revised edition published in 2023, precisely so the institution can sustain and develop the platform long term without depending on a single supplier. The choice of a standard data model is the decision that most determines whether the project outcome is still yours in ten years.
Frequently asked questions
Document AI extraction, NER and NED
What is the difference between NER and NED?
Why do document extraction projects fail in production?
Batch architecture and language
Should document processing run in batch or in real time?
Do generic models work on documents in Portuguese?
Prerequisites and regulation
What do you need before contracting a document AI project?
Does AI document extraction fall within the AI Act?
