The challenges of “useful” OCR

The National Archive’s digitisation project, British Governance in the 20th century – Cabinet Papers, 1914-1975, has been grappling with issues of “useful” OCR. It might be stating the obvious, but accurate OCR is as useful as the search results it produces.

If OCRd text consistently misspells particularly relevant key words for retrieving certain documents, than the search results against these key words will not always bring up appropriate documents, and will lack in accuracy.

For the National Archives, it was not enough to establish a range of acceptable OCR performance levels purely from a quantitative point of view, eg OCR performance accuracy should not be below 88%. This is because if the remaining 12% of text that is not accurate includes particularly relevant key words for retrieving a certain document that users are likely to search by, the discovery of that document is impeded or made less likely. Eg, if the word “submarine” is particularly relevant to the subject of a document, and it’s consistently misspelt by the OCR software, the likelihood of discovering that document is less than if another, less relevant, word, had been misspelled. So, even matching an established minimum percentage of performance (eg 88%), does not necessarily mean that search results will be accurate or useful.

The National Archives are also adopting a more qualitative approach to run alongside the quantitative one described above. They are concentrating on identifying the most relevant and frequently misspelt “key” words across all of the OCRd documents. They are then planning to run a global “search and replace” to reinstate the correctly spelt words.

Although this will have marginal effect on the overall accuracy ratings, this will increase the usefulness of OCR to the end user.