Virtual Reference View Generation for CBIR-based Visual Pose Estimation

At ACM Multimedia 2012, we present a method that enables meter-accurate indoor positioning using visual information recorded by a smartphone’s camera. The position and orientation of the device is estimated by comparing the camera images to a database of previously computed virtual views. This comparison is carried out by an optimized image search engine and can be done within miliseconds. The paper covers the view generation process and explains how the virtual views are used for visual localization.

In a nutshell, the approach employs a novel combination of a content-based image retrieval engine and a method to generate virtual viewpoints for the reference database. In a preparation phase, virtual views are computed by transforming the viewpoint of images that were captured during the mapping run. The virtual views are represented by their respective bag-of-features vectors and image retrieval techniques are applied to determine the most likely pose of query images. As virtual image locations and orientations are decoupled from actual image locations, the system is able to work with sparse reference imagery and copes well with perspective distortion.

Please check out our ACM Multimedia 2012 paper for all the details.

This is a live demonstration of the visual indoor localization & navigation system developed for the project NAVVIS at the Institute for Media Technology at Technical University Munich:

Download: 720p WMV 1080p MP4

The video shows our lab demonstrator running on an Android phone. By analyzing the camera images for distinctive visual features, the position and orientation of the smartphone is recognized and displayed on the map. For this demonstration, only visual information has been used – other localization sources have not been used to improve the vision-based results. In order to compare localization accuracy, Android’s network-based position estimate (Wifi and cellular networks) is displayed.

This entry was posted in Navvis. Bookmark the permalink.

Comments are closed.