[Colloq] Hiring talk: Design Techniques for Crowdsourcing Complex Tasks, Feb 27 10:30AM, 366WVH

[Francoise Niang]

Title:  Design Techniques for Crowdsourcing Complex Tasks

Speaker:  Edith Law
Date: 2/27/2014 at 10:30, 366WVH

Abstract:
Human computation (a.k.a. crowdsourcing) systems are theoretically interesting because they challenge the way we currently think about and build intelligent systems.  We now have to design the system to take into account factors that affect how people compute, including their motivation, cognitive limitations and expertise.  Having access to both automated algorithms and many human computers also means that, as system designers, we must explicitly reason about the division of labor -- between novices, experts, and machines -- that will lead to the best computational outcomes.

There are numerous examples of human computation systems achieving remarkable feats -- massively and rapidly labeling images (e.g., the ESP Game), digitizing books (e.g., reCAPTCHA), folding proteins (e.g., FoldIt), translating text (e.g., Duolingo).   Yet, many of the problems tackled through crowdsourcing are simple, in that they require only basic perceptual abilities and common-sense knowledge, or that they can be handled by independent workers each having only a local view of solution.   In this talk, I will describe several general design techniques for crowdsourcing complex tasks and specific examples of their use in developing a variety of human computation systems, including games with a purpose, and social computing platforms for planning and text summarization.    

Can we extend existing crowdsourcing models to handle tasks that require substantially more expertise, such as research tasks involving the collection, annotation and analysis of scientific data?  How can we lower the barrier of entry for scientists, who are domain experts but not necessarily technically savvy or familiar with crowdsourcing, to use crowdsourcing as a tool for their research?  I will conclude by describing my research agenda on mixed-expertise crowdsourcing in the scientific domain, and a citizen science platform and research infrastructure, called Curio, for exploring an entirely new space of complex problems that can benefit from leveraging contributions from expert communities and non-expert crowds.  


Biography:
Edith Law is a CRCS postdoctoral fellow at the School of Engineering and Applied Sciences at Harvard University. She graduated from Carnegie Mellon University in 2012 with Ph.D. in Machine Learning, where she studied human computation systems that harness the joint efforts of machines and humans.  She is a Microsoft Graduate Research Fellow, co-authored the book "Human Computation" in the Morgan & Claypool Synthesis Lectures on Artificial Intelligence and Machine Learning, co-organized the Human Computation Workshops, and helped create the first AAAI Conference on Human Computation and Crowdsourcing. Her work on games with a purpose and large-scale collaborative planning has received best paper honorable mentions at CHI. 

Host: Alan Mislove