K.U.Leuven > ESAT > PSI > Visics > Research > Topics > Item 2.2

Affinely invariant neighbourhoods and applications

L. Van Gool, T. Tuytelaars

We developed a system to automatically find correspondences between two different images of the same object or scene independent of viewpoint or illumination changes. To this end, we introduced the concept of affinely invariant neighbourhoods. These are local image patches that are constructed around anchor points in each image independently. The crux of the matter is that their shape depends on the underlying image intensities in such a way that they always cover the same physical part of the object (provided that the neighbourhood can be considered flat).

Once such neighbourhoods have been extracted, they can be described using affine moment invariants. Based on the invariants, the points (or neighbourhoods) found in one image can be matched quite efficiently with their corresponding points in other images, even if the position if the camera has changed drastically in between the two shots.

This can be used to compute the relation between two images of an object or a scene (a vital step for e.g. making a 3D reconstruction). Other applications of the invariant neighbourhoods range from recognizing and localizing relevant objects in an image, over content-based image retrieval from a database of images to visual homing of a mobile robot.

Due to the affine invariance, corresponding neighbourhoods are extracted in both images below in spite of the large change in viewpoint.

Due epipolar geometry between both images below, computed based on matched invariant neighbourhoods.