Too Long; Didn't Read

We studied the multifaceted problem of designing a crowdsourcing mechanism that efficiently and accurately ranks a set of items using pairwise comparisons from rational agents. We based our approach on the classic principal-agent model from contract theory. To distribute comparisons among agents, we used an unexpected connection to the social golfer problem which allowed us to simultaneously minimize the agents’ workload and ensure enough agents evaluate each comparison.