K.U.Leuven > ESAT > PSI > Visics > Research > Topics > Item 4.1

Compression for immersive visualization (SUPERVISIE)

L. Van Gool, L. Van Eycken, B. Deknuydt

A first topic in this project is the efficient coding of non-static texture in 3D scenes, when the coder knows the observers' viewing positions through a back channel. A previously developed video codec is adapted to incorporate additional features to take into account scene and viewpoint dependent parameters. Currently the distance to the camera, and the tilt and slant angle to the viewing plane are used to determine required coding resolution. Ongoing research is to extend this codec for dynamic textures projected on non-flat objects (e.g. the model of a talking head). In this case it is no longer appropriate to use an arbitrary segmentation, like rectangular blocks. The 3D viewing parameters are now different for every primitive (triangle, polygon, ...) used by the 3D mesh. But since the decoder also knows the underlying mesh, even a complex segmentation does not require any additional bits. The viewing parameters (visibility, distances to and angles with the viewing plane) are also deductible at the decoder side, so no additional bit rates are required. This extra knowledge allows the encoder to determine exactly the required resolutions for each part of the texture, and code it just well enough. Savings of typically a factor 5 in bit rate, in comparison with MPEG-type coding, are achievable. As an example, the false colour image on this page shows the required vertical texture resolution.

A second topic is the simplification (decimation) of 3D mesh, using texture information. Lots of decimation algorithms are described in literature, but up to now, most of them use a geometry based quality measure. However, 3D objects are usually represented with an associated texture, which can greatly mask imperfections in the geometry. After a literature search for a flexible and fast decimation algorithm, Garland and Heckbert's approach was implemented and extended with texture information. Both straightforward additions of the error measure, like quadratic differences, as more complex distances in human visual system aware colour spaces are being evaluates.