PUMA
Istituto di Scienza e Tecnologie dell'Informazione     
Freire A., Macdonald C., Tonellotto N., Ounis I., Cacheda F. Hybrid query scheduling for a replicated search engine. In: ECIR 2013 - Advances in Information Retrieval. 35th European Conference on IR Research (Moscow, Russia, 24-27 March 2013). Proceedings, pp. 435 - 446. Pavel Serdyukov, Pavel Braslavski, Sergei O. Kuznetsov, Jaap Kamps, Stefan Rüger, Eugene Agichtein, Ilya Segalovich, Emine Yilmaz (eds.). (Lecture Notes in Computer Science, vol. 7814). Springer, 2013.
 
 
Abstract
(English)
Search engines use replication and distribution of large indices across many query servers to achieve efficient retrieval. Under high query load, queries can be scheduled to replicas that are expected to be idle soonest, facilitated by the use of predicted query response times. However, the overhead of making response time predictions can hinder the usefulness of query scheduling under low query load. In this paper, we propose a hybrid scheduling approach that combines the scheduling methods appropriate for both low and high load conditions, and can adapt in response to changing conditions. We deploy a simulation framework, which is prepared with actual and predicted response times for real Web search queries for one full day. Our experiments using different numbers of shards and replicas of the 50 million document ClueWeb09 corpus show that hybrid scheduling can reduce the average waiting times of one day of queries by 68% under high load conditions and by 7% under low load conditions w.r.t. traditional scheduling methods.
URL: http://link.springer.com/chapter/10.1007%2F978-3-642-36973-5_37
DOI: 10.1007/978-3-642-36973-5_37
Subject Information Retrieval
H.3.3 Information Search and Retrieval


Icona documento 1) Download Document PDF


Icona documento Open access Icona documento Restricted Icona documento Private

 


Per ulteriori informazioni, contattare: Librarian http://puma.isti.cnr.it

Valid HTML 4.0 Transitional