How to run a ranked-choice vote that minimizes penalties to unranked candidates?

I want to run a vote to pick favorite video games, but I think this applies to many other voting domains.

The scenario is people rank games they like from favorite to least favorite. I would like to find a community aggregate list of favorites ranked from most to least favorite. The problem is that, given a list of candidates, most people have not played all of them and will only rank the games they have played.

An example forum where people are ranking games.

I would like a ranked-ballot system that considers un-ranked candidates as having no preference and generates a ranked list of results.

I am using a Condorcet voting site that has many voting system options:

You can look at the head-to-heads between two candidates only on the ballots where both are ranked. Then you can apply a Condorcet method (e.g. Ranked Pairs) to these head-to-heads. You still need to decide what counts as the defeat strength though (e.g. winning votes or margins).

For example, A might be ranked top of 100 ballots, with B unranked on these ballots. There are only 5 ballots where both A and B are ranked, and B is ahead 3-2. So the 100 ballots putting A top and B nowhere are ignored, and it’s a 3-2 head-to-head win to B. So you can count that as a margin of 1, or 3 winning votes, or something else.

1 Like

I’m thinking that, in the spirit of Majority Denominator for scored methods, you could have a certain percentage of pairwise matchups between two options where only one of the options is ranked on the voter’s ballot be counted as if the unranked option was ranked lower than the ranked option. As an example, if you have 100 voters that rank:

51 A
20 A>B
29 B>A

If you just ignore ballots that didn’t rank both candidates, B beats A 29 to 20.
But, if we take a majority of those 51 A voters as “actually” having voted A>B, then you get:

25 A
46 A>B
29 B>A

A beats B 46 to 29.

Since you’re looking to minimize penalties for unranked candidates, a better percentage might be 20% to 30%. Assuming 30%, the above becomes:

35 A
36 A>B
29 B>A

A wins 36 to 29.
Maybe you might even want to shave it down to 10%, in which case it’d be B beats A 29 to 26.
If you don’t want even that much, with 1% it becomes B beats A 29 to 21, so maybe take something between 5% and 10% over 5%?
Edit: You could also ignore ranked matchups where only a minority of voters ranked both options, or decrease the amount that such matchups count towards the overall ranking of either option proportional to the percentage of voters who ranked either. At a low enough level, you might want to take “small sample sizes” into account and ignore the result.

It might be difficult to compare games across genres no matter what you choose since presumably the games voters have played will be similar to each other.

For defeat strength of Y vs X, could you use the z-score for the alternative hypothesis p[X>Y]>.5? (Where the sample is the ballots giving ranks to both X and Y.)

1 Like

After thinking about this and several discussions (here and elsewhere) I’m developing and idea of an alternative to ranked ballots. Something like ranked ballots mixed with approval. Unfortunately this means existing ranked votes cannot be used.

Voters would rank their ballots and then specify which position has the lowest approved/enjoyed candidate. Unranked candidates will be inserted below this position (all equally) with the other rankings shifted down.

Try to use DV in ranking format, that is:

  • people vote ranking candidates.
  • normalize the ranking votes making them 100 points each.
    Vote 1) 1st, 2nd, 3rd, 4th → 40, 30, 20, 10
    Vote 2) 1st, 2nd → 67, 33
  • use the DV count which gives you as many winners as you like (it also tells you how much better one winner is than the others).

DV wasn’t born as a ranked ballot system but I think it works better than many other methods in this context.