Temporal
Video Profile for Event Detection and Behavior Extraction
PI: J. Zheng (Department
of Computer and Information Science, IUPUI)
Video has been used to record motion and events in daily life. Video
database contain tremendous amount of visual information and current
video sharing infrastructure such as Youtube has greatly extended the
video sharing capability. However, video is still a medium difficult to
search and understand before watching it. It is particularly
challenging to index and retrieval video in its organizing, reusing,
and analysis, because of its spatial and temporal data format and a
large data size. Creating an effective visualization style at a finer
level than key frames is largely demanded for indexing and analysis
that facilitates the wide use of video. Many works have been done in
manual indexing of video, but none of them have realized the goal of
automatic visualization and retrieval based on visual contents. The
reason is a semantic gap between low level pixels and high level
symbols that is hard to bridge by data learning and ad-hoc inference.
Theoretical modeling at each level of data abstraction has to be
carried out for meaningful information filtering and extraction in the
video clips, streams, and database. This work has the objectives as
follows: 1) Modeling the camera operations in terms of basic camera kinematics in order to tackle all types of video. 2) Modeling the optical flow effects observable in the video so as to separate motion and scenes in video clips. 3)
Designing a representation and algorithms to condense 3D video volume
to motion traces and temporal profiles of video in order to tell what
is in the video, when it happens, and how it moves, before watching the
video itself. 4) Extracting human behavior from the motion traces and profiles in a large database of ego-centric video. 5)
Modeling cognitive learning process by finding human behaviors based on
video profiles and traces during the child-parent interaction. The
results of this work will benefit the browsing of media data,
retrieving personal video events, and the analysis of video recording
experiments in cognitive and medical research.
