Unsupervised deep autoencoder-based reconstruction for ink mismatch detection in hyperspectral document images

Abstract

Ink analysis is crucial for finding ink mismatches in handwritten documents to determine their authenticity and identify potential forgery. Traditional chemical-based methods like thin layer chromatography are effective for this purpose, but they are destructive, irreversible, time-consuming, and sensitive to environmental factors. Hyperspectral document images (HSDIs) which capture the spectral information in multiple bands can reveal the material composition, thereby allowing the identification of different inks by their distinct spectral characteristics, even when the inks visually appear to be the same colour. Hyperspectral imaging thus shows great promise for forensic document analysis and authentication. Despite its potential, research in this area is still emerging. Existing HSDI-based methods show promise in detecting ink mismatches but these methods often require prior spectral information and ground truth data for the ink pixels, limiting their practical applicability. Unsupervised methods present a solution by removing the need for prior information. This work proposes a novel unsupervised ink mismatch detection method in HSDIs using deep learning-based hyperspectral analysis. The proposed method formulates ink mismatch detection as a decision problem that determines whether a deep autoencoder trained on the spectral features of specific ink pixels would be able to reconstruct the spectral features of unseen complementary ink pixels. The proposed framework works in a fully unsupervised manner, learning the spectral representations directly from data without any labeled samples or prior spectral information of the inks present. Unlike most existing methods which are supervised, the proposed unsupervised method is more practically applicable for real world forensic document analysis. Experimental results on a publicly available HSDI dataset demonstrate the superiority of the proposed method over existing methods in identifying ink mismatches in potentially fraudulent document images. © The Author(s) 2025.