K.U.Leuven > ESAT > PSI > Visics > Research > Topics > Item 4.5

Modelling user-feedback for Content-Based Image Retrieval

L. Van Eycken, E. Pauwels, G. Caenen, G. Frederix

The explosive growth of digital image databases has highlighted the need for accurate and efficient content-based image retrieval (CBIR). A major difficulty is due to the fact that there is no canonical way to capture the visual content that is encapsulated in an image. Indeed, the definition of content is intricately tied up with the underlying visual appreciation and goals of the user, and will consequently vary from occasion to occasion. Most CBIR search-engines therefore opt to "keep the user in the loop" by regularly requesting his feedback. Hence the interest in intelligent interfaces that prompt the user for feedback and then try and capitalize on it to expedite the search-action. More precisely, the feedback is harnessed to estimate for each image in the database the likelihood of its relevance to the user, whereupon the most promising candidates are displayed for further inspection and feedback. This procedure is then iterated as often as necessary to locate the target image.

The specific form in which feedback is solicited varies from interface to interface. In the earliest versions (e.g., QBIC), the user could express perceptual preferences by adjusting sliders that govern the relative weights of pre-specified image-features. However, it turned out that the sheer number of features and their often abstract meaning makes it difficult for the user to keep an overview and plan ahead. To circumvent these problems, we are developing an interface that focuses on images rather than features. The user is no longer prompted to specify individual feature values or weights, but can directly express his (global) preference of one image relative to the rest. More precisely, the user is shown a number of images that the interface deems to be very relevant to the search and the user can express whether he agrees or not. This generates a number of examples and counter-examples that are used by an underlying inference engine to estimate the parameters of a probabilistic model for image relevance. The updated relevance probability model is then harnessed to produce a new sample of images for the next iteration cycle. Clearly, compared to the original feature-centred interaction, this type of interaction is perceptually more transparent and appealing. The design of the inference engine is one of the main research topics in this project.