K.U.Leuven > ESAT > PSI > Visics > Research > Topics > Item 1.1

Photo-realistic modelling of three-dimensional surfaces from camera images

L. Van Gool, M. Pollefeys, M. Vergauwen, J. Tops, F. Verbiest

A 3D surface modelling system was developed that reconstructs photo-realistic 3D models of a scene from images alone. The 3D scene is observed from multiple viewpoints by freely moving a camera around the object. No restrictions on camera movement and internal camera parameters like zoom are imposed, as the camera pose and intrinsic parameters are calibrated from the sequence. The only restrictions on the scene content are the rigidity of the scene objects and opaque, piecewise-smooth object surfaces. The approach operates independently of object scale and requires only a single low-cost consumer photo or video camera.

The modelling system described here uses a four-step approach. First, salient feature points are extracted from the views. Corresponding feature points are matched using a robust algorithm that also yields relations between the views. The second step consists of jointly computing the 3D position of the feature points and the pose and intrinsic parameters of the camera. Next, consecutive images of the sequence are treated as stereoscopic image pairs and dense correspondence maps are computed by area matching. Dense and accurate depth maps are computed by linking together all correspondences over the viewpoints. Finally, the depth maps are converted to triangular surface meshes that are texture mapped for photo-realistic appearance. The resulting surface models can be used for visualization or measurements purposes. Applications can be found in many different fields such as archaeology, architectural conservation, forensic science, robot navigation and many more.

The above figure shows a reconstruction of 3D models from images. The left image shows one of the five original images of an Indian temple and the right one shows a view of the 3D model. A movie of the 3D model is available as temple.mpg.