Home / AI Research / EgoCaptioner: Towards Comprehensive and Fine-Grained Egocentric Video Captioning
EGAI Research
EgoCaptioner: Towards Comprehensive and Fine-Grained Egocentric Video Captioning
by KlingTeam · Yuqi Tang, Yang Shi, Yuran Wang, Tengfei Liu, Rundong Wang, Zongyang Qiu
Egocentric video captioning requires fine-grained detail and accurate timing, yet current multimodal large language models (MLLMs) often miss brief actions, confuse their order, or misinterpret the wearer's motion. Existing caption formats also offer limited support for combining detailed narration with precise timing. We present EgoCaptioner, which pairs a video-level overview of the setting, participants, and the wearer's overall activity with timed steps whose narration adapts to varying perceptual demands and marks key moments at specific timestamps. We develop a coarse-to-fine annotation pipeline that establishes event structure before expanding step descriptions, yielding 50,000…
Upvotes25
Comments2
Published18 Nov 2026