[Colloq] Thesis Proposal Presentation

[Nicole Bekerian]

The College of Computer and Information Science presents:

Speaker: Shahzad K. Rajput
Date: July 7th, 2011 (Thursday) at 9am
Where: WVH 366

Thesis title:  
"A Nugget-based Test Collection Construction Paradigm"

Abstract:
"The problem of building test collections is central to the development of information retrieval systems such as search engines. The primary use of test collections is the evaluation of IR systems. The widely employed "Cranfield paradigm" dictates that the information relevant to a topic be encoded at the level of documents, therefore requiring effectively complete document relevance assessments. As this is no longer practical for modern corpora, numerous problems arise, including scalability, reusability, and applicability.

We propose a new method for relevance assessment based on relevant information, not relevant documents. Once the relevant information is collected, any document can be assessed for relevance, and any retrieved list of documents can be assessed for performance. Starting with a few relevant "nuggets" of information manually extracted from existing TREC corpora, we implement and test a method that finds and correctly assesses the vast majority of relevant documents found by TREC assessors, as well as up to four times more additional relevant documents. We then show how these inferred relevance assessments can be used to perform IR system evaluation. Our main contribution is a methodology for producing test collections that are highly accurate, more complete, scalable, reusable, and can be generated with similar amounts of effort as existing methods, with great potential for future applications."

Thesis Committee:

Javed A. Aslam (Advisor)
Carole D. Hafner
Alan Mislove
Virgil Pavlu
Ian Soboroff (External Examiner)

-- 



Nicole Bekerian 
Administrative Assistant 

Northeastern University 
College of Computer and Information Science 
360 Huntington Ave. 
202 West Village H 
Boston, MA 02115 

Phone: 617.373.2462 
Fax: 617.373.5121