The Living Labs for Information Retrieval Evaluation (LL4IR) CLEF Lab
September, 2015, Toulouse, France
[Registration closes: 30 April, 2015]
===========================================================================================
This lab provides a benchmarking platform for researchers to evaluate their ranking systems in a live setting with real users in their natural task environments. The lab acts as a proxy between commercial organizations (live environments) and lab participants (experimental systems).
The lab focuses on three use-cases (using a shared API):
Task 1: product search (on the REGIO JÁTÉK e-commerce site)
Task 2: local domain search (on the University of Amsterdam's website)
Task 3: web search (through Seznam, a major commercial web search engine)
-------------------------------------------
What is in it for you as a lab participant?
-------------------------------------------
- Access to privileged commercial data (click through, etc.).
- Access to new static TREC-style collections.
- Access to the living labs evaluation environments.
- Opportunity to test your IR systems on real live systems with real live users.
-----------------------
Current Important dates
-----------------------
31 Mar, 2015: Training period ends
1-15 Apr, 2015: Uploading test runs
15 Apr, 2015: Testing period begins
15 May, 2015: Testing period ends
-------------------
Further Information
-------------------
Further information is available on the lab website (http://living-labs.net/clef-lab/) or by emailing the lab organisers.
--------------
Lab Organisers
--------------
Krisztian Balog – University of Stavanger, Norway (krisztian.balog (at) uis.no)
Liadh Kelly – Trinity College Dublin, Ireland (liadh.kelly (at) tcd.ie)
Anne Schuth – University of Amsterdam, The Netherlands (anne.schuth (at) uva.nl)
No comments:
Post a Comment