FrameBoost: Advanced Video Analytics With Inference Trigger Frame Selection via Tracking Error Estimation
J. Yang, Sunwook Hwang, Jeongjun Park, Saewoong Bahk
For real-time video analytics, the inference trigger frame selection problem is crucial. While continuous video streams offer rich data, processing every frame is computationally intensive. Thus, modern systems strategically analyze the frames to choose frames for inference, i.e., inference trigger frames (ITF). Between ITFs, they use tracking to leverage temporal coherence. Consequently, the ITF selection problem functions as the cornerstone technology for video analytics in terms of both accuracy and efficiency. Additional to selecting the critical frames for system accuracy, a major requirement for the ITF selection algorithms is ‘efficiency.’ ITF selection algorithms should be able to extract the informative frames while imposing minimal computational overhead to the analytics pipeline. While current solutions, such as ‘frame differencing’ methods, provide promising results, they still fall short of fully addressing this issue.We point out that using frame difference value as the metric can mislead the re-detection necessity, thus falling short in choosing the optimal frame sets for inference. We introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FrameBoost</i>, an intelligent ITF selection method that adopts aggregate tracking error (ATE) as its metric, leveraging object-wise IoU predictions. We study the key factors controlling the IoU prediction performance and propose a lightweight solution that effectively addresses the aforementioned limitations. Through extensive evaluations measuring both accuracy and computational efficiency, we demonstrate that FrameBoost delivers superior performance compared to existing approaches.
https://doi.org/10.1109/access.2025.3558251
Computer science
Analytics
Inference
Frame (networking)
Selection (genetic algorithm)
Artificial intelligence
Tracking (education)
Data mining
Pattern recognition (psychology)
Telecommunications
상세 정보 바로가기