Leveraging vision language models for digit recognition on digital measuring devices: a comparative analysis

Abstract

Large language models (LLMs) with multimodal input capability have been demonstrated in various applications. Vision language models (VLMs), which were developed from LLMs, can process and learn both visual and textual data simultaneously, enabling them to generate descriptive text from images. This study explores the use of several VLM models for the seven-segment digit recognition of a digital measuring device compared with other baseline optical character recognition (OCR) methods, such as Tesseract OCR, EasyOCR, PaddleOCR, KerasOCR, and YOLO11, with or without the integration of you only look once 11 (YOLO11) for LCD screen cropping. The results showed that the Gemini model outperformed conventional OCR methods and other VLM models with an average per-digit-recognition (PDR) accuracy of 98.10% (?=0.36%) and full sequence reading (FSR) accuracy of 93.85% (?=1.29%). The performance evaluation across the distance variation also demonstrates high accuracy for the VLM result with stable results. Simultaneously, YOLO11 exhibits degradation at longer distances due to difficulty in detecting small decimal digits, whereas the traditional OCR method yields inconsistent results. This study showed promising results for the further development, improvement, and deployment of VLM models in the automatic seven-segment digit recognition task on the edge device.