ORT: Unintended Text Recognition from Eyeglass Reflections in Video Conferencing Environments
Jieun Kim, Youngjoo Park, Rokyung Kwon, Jimin Han, Hoorin Park
Since the COVID-19 pandemic, the rise of video con-ferencing has introduced new information security challenges, particularly the unintentional leakage of sensitive information via reflected text on eyeglass surfaces captured by webcams. While prior studies have shown the possibility of webcam peeking attacks, they typically rely on manual analysis or costly equipment, limiting scalability. To address these challenges, we propose Optical Character Recognition (OCR) of Reflected text, called ORT, an automated attack model that extracts re-flected text using advanced OCR techniques. ORT is composed of customized preprocessing steps and optimized Scene Text Detection (STD) and Scene Text Recognition (STR) models, with the STD model retrained on a custom reflected text dataset to enhance performance under conditions such as blur, low contrast, and distortion. We evaluated ORT in various video conferencing environments, taking into account variables such as lighting, text size, background color, and the presence or absence of prescriptions for eyeglasses. The results show high recognition accuracy, demonstrating that ORT significantly reduces adversary effort and raises the threat level of webcam peeking attacks.
https://doi.org/10.1109/acsac67867.2025.00059
Preprocessor
Text recognition
Limiting
Optical character recognition
Videoconferencing
Facial recognition system
Character (mathematics)
Information leakage
상세 정보 바로가기