Highlights at iM2CS

School of Science Institute for
Mathematical Modeling and
Computational Science

  iM2CS home       research     people     seminars     highlights     opportunities for students     funding opportunities  

Temporal Video Profile for Event Detection and Behavior Extraction

PI: J. Zheng (Department of Computer and Information Science, IUPUI)

Video has been used to record motion and events in daily life. Video database contain tremendous amount of visual information and current video sharing infrastructure such as Youtube has greatly extended the video sharing capability. However, video is still a medium difficult to search and understand before watching it. It is particularly challenging to index and retrieval video in its organizing, reusing, and analysis, because of its spatial and temporal data format and a large data size. Creating an effective visualization style at a finer level than key frames is largely demanded for indexing and analysis that facilitates the wide use of video. Many works have been done in manual indexing of video, but none of them have realized the goal of automatic visualization and retrieval based on visual contents. The reason is a semantic gap between low level pixels and high level symbols that is hard to bridge by data learning and ad-hoc inference. Theoretical modeling at each level of data abstraction has to be carried out for meaningful information filtering and extraction in the video clips, streams, and database. This work has the objectives as follows:
1) Modeling the camera operations in terms of basic camera kinematics in order to tackle all types of video.
2) Modeling the optical flow effects observable in the video so as to separate motion and scenes in video clips.
3) Designing a representation and algorithms to condense 3D video volume to motion traces and temporal profiles of video in order to tell what is in the video, when it happens, and how it moves, before watching the video itself.
4) Extracting human behavior from the motion traces and profiles in a large database of ego-centric video.
5) Modeling cognitive learning process by finding human behaviors based on video profiles and traces during the child-parent interaction.
The results of this work will benefit the browsing of media data, retrieving personal video events, and the analysis of video recording experiments in cognitive and medical research.