Description

Query performance predictors are commonly evaluated by reporting correlation coefficients to denote how well the methods per-form at predicting the retrieval performance of a set of queries. Despite the amount of research dedicated to this area, one aspect remains ne-glected: how strong does the correlation need to be in order to realize an improvement in retrieval effectiveness in an operationalsetting? We address this issue in the context of two settings: Selective Query Ex-pansion and Meta-Search. In an empirical study, we control the quality of a predictor in order to examine how the strength of the correlation achieved, affects the effectiveness of an adaptive retrievalsystem. Recently a lot of work was performed towards enabling keyword search in databases. However, database search engines are mostly adapted to the queries manually created by users. In case user’s information need is expressed in terms of a document, we need to create algorithms that automatically extract keyword queries from the available data and map them to the database content. Keyword queries on databases provide easy access to data, but often suffer from low ranking quality, i.e., low precision and/or recall, as shown in recent benchmarks. It would be useful to identify queries that are likely to have low ranking quality to improve the user satisfaction. For instance, the system may suggest to the user alternative queries for such hard queries. In this paper, we analyze the characteristics of hard queries and propose a novel framework to measure the degree of difficulty for a keyword query over a database, considering both the structure and the content of the database and the query results.