Image Annotation by Semi-Supervised Clustering Constrained by SIFT Orientation Information

Methods developed for image annotation usually make use of region clustering algorithms. Visual codebooks are generated from the region clusters of low level features. These codebooks are then, matched with the words of the text document related to the image, in various ways. In this paper, we supervise the clustering process by using the orientation information assigned to each interest point of Scale-invariant feature transform (SIFT) features to generate a visual codebook. The orientation information provides a set of constraints in a semi-supervised k-means region clustering algorithm. Consequently, in clustering of regions not only SIFT features are normalized along the dominant orientation, but also orientation information itself is used. Experimental results show that image annotation with added orientation information by semi-supervised clustering is more successful compared to the one that uses SIFT features alone. The proposed algorithm is implemented in a parallel computation environment.