PUMA
Istituto di Scienza e Tecnologie dell'Informazione     
Peters C. Multilingual information access: the contribution of evaluation. In: IWRDL 2006 - 2006 International Workshop on Research Issues in Digital Libraries (Kolkata, India, 12-15 December 2006). Proceedings, ACM, 2006.
 
 
Abstract
(English)
Since evaluation of cross-language information retrieval systems began at TREC in 1997 and NTCIR in 1998 and, in particular, with the launch of the Cross-Language Evaluation Forum (CLEF) in 2000, considerable progress has been made in this particular sector of IR. Advances can be considered in two stages. The first stage regarded in particular the development of text retrieval systems from simple so-called "bilingual" systems in which a query in one language is used to search a document collection in another to truly "multilingual" retrieval systems where a query in one language can find relevant results from a collection of documents in multiple languages. In the second stage, the focus was no longer just on multilingual document retrieval but was diversified to include different kinds of text retrieval across languages (e.g multilingual question answering) and retrieval on different kinds of media (e.g. collections containing images or speech). However, although the results from the research perspective have been interesting, there has been little real take-up by the applications communities. In the paper we describe the results achieved by CLEF over the years and propose a third stage for multilingual system evaluation which gives far more attention to questions regarding usability and user satisfaction but also provides ways for the results achieved to be transferred to the operational context.
URL: http://portal.acm.org/toc.cfm?id=1364742&type=proceeding&coll=GUIDE&dl=GUIDE&CFID=12587286&CFTOKEN=61476350
DOI: http://doi.acm.org/10.1145/1364742.1364761
Subject Multilingual Information Access
Evaluation
Languages
Test Collections
H.3.3 Information search and retrieval
H.3.4 Systems and Software. Performance evaluation (efficiency and effectiveness)


Icona documento 1) Download Document PDF


Icona documento Open access Icona documento Restricted Icona documento Private

 


Per ulteriori informazioni, contattare: Librarian http://puma.isti.cnr.it

Valid HTML 4.0 Transitional