Content-based Image and Video Retrieval using HMMs
Speaker:Arnab Ghoshal
Institution:Johns Hopkins
Abstract:
In this work we propose a hidden Markov model based method for automatic annotation of images with keywords from a generic vocabulary of concepts or objects for the purpose of content-based image and video retrieval. An image or a video frame, represented as sequence of feature-vectors characterizing low-level visual features such as color, texture or oriented-edges, is modeled as having been stochastically generated by a hidden Markov model. The parameters of the model are estimated from a set of manually annotated (training) keyframes. Each keyframe in a search collection is then automatically scored for the presence of a concept. This annotation supports content-based search of a image or video collection via keywords. Moreover, adaptation of hidden Markov model (HMM) parameters to individual speakers is known to provide considerable improvements over speaker-independent speech recognition systems. The current work applies this idea of model adaptation to a content-based video retrieval system that uses HMMs, with different sources of video treated analogously to different speakers. Empirical retrieval results are presented on the NIST TRECVID 2003, 2005 and 2006 benchmark collections.