Abstract
This paper evaluates the performance of different voting and probability aggregation methods using a dataset of 451,200 probability judgments from 376 participants on 1200 general knowledge claims. We investigate the Condorcet Jury Hypothesis, according to which majority voting tends to yield outcomes more accurate than the average individual. Furthermore, incorporating Locke’s Thesis to translate probabilistic judgments into categorical beliefs, we compare six aggregation methods: majority, strict majority, unanimity, strict unanimity, probability averaging, and log-odds averaging. Our analysis reveals clear accuracy–decisiveness trade-offs for all methods. Majority voting achieves high decisiveness but moderate accuracy; unanimity rules can produce high accuracy when decisive, but, especially in their strict form, do so at the cost of frequent indecisiveness. Probability aggregation methods often do best in balancing accuracy and decisiveness. Among our main findings is that strict majority voting, a cornerstone of many democratic institutions, is often outperformed on both accuracy and decisiveness simultaneously by other methods. We also find that judgment diversity contributes to group accuracy: groups whose members give less correlated probability judgments tend to achieve higher accuracy.
