K.U.Leuven > ESAT > PSI > Visics > Research > Topics > Item 1.6

Virtual guide

L. Van Gool, P. Wambacq, M. Pollefeys, D. Martens

The virtual guide is set up as an integrated demonstrator for natural speech recognition and virtual reality navigation of image-based 3D model. The core of the system is an agent, represented by an animated mask, which can be addressed by the user, in a way similar to a normal face-to-face conversation. Therefore, beyond recognizing the words, the system also needs to understand what the users request means and what the appropriate response is. As part of this project research is being carried out to develop advanced multi-modal human-computer interaction.

The demonstrator that has been build allows a virtual visit to the archaeological site of Sagalassos (Turkey). The virtual models of the landscape and some of the monuments where automatically reconstructed from images. The model was then augmented with CAD models of the ancient buildings provided by archaeologists. The name and position of the points of interest where thought to the virtual guide.

The figure below shows the virtual guide in the landscape of Sagalassos. A movie showing the interaction between the virtual guide and a user is available as eamos.mpg.