Repeated judgements in elicitation tasks: Efficacy of the MOLE method

Abstract

Recent work has demonstrated the benefits of repeated judgments in improving the accuracy of estimates – both independent and repeated, individual judgments. The More- Or-Less-Elicitation (MOLE) method has previously been shown to improve accuracy and precision of elicitation over single judgment elicitations (Welsh, Lee, & Begg, 2008). In this paper, we show the MOLE method delivers superior gains, in terms of both accuracy and calibration, over repeated-judgment elicitation, while taking significantly less time and not requiring filler tasks to distract participants from previous estimates. We argue that the structure of the MOLE method acts in the same manner as repeated judgment, resulting in multiple searches across the relevant space, without the associated problems of standard repeated judgments methods, such as additional time and participant recall and repetition of prior estimates.