ITA Software's Technical Seminar Series

Talk Topic: Turkit: Tools for Iteratie Tasks on Mechanical Turk
Greg Little, MIT CSAIL
July 15, 2009

Abstract

Mechanical Turk (MTurk) is an increasingly popular web service for paying people small rewards to do human computation tasks. Current uses of MTurk typically post independent parallel tasks. This paper explores an alternative iterative paradigm, in which workers build on or evaluate each other's work. We describe TurKit, a new toolkit for deploying iterative tasks to MTurk, with a familiar imperative programming paradigm that effectively uses MTurk workers as subroutines, such as the comparison function or a sorting algorithm. The toolkit handles the latency of MTurk tasks (typically measured in minutes), supports parallel tasks, and provides fault tolerance to avoid wasting money and time. We present a variety of iterative experiments using TurKit, including image description, copy editing, handwriting recognition, and sorting.

Bio

Greg did his undergrad in Computer Science at Arizona State University. He has a Masters in Computer Science at MIT, and is continuing work toward a Ph.D. there. His interests are broad, but tend to lie at the intersection of computer science and psychology.